Results 1  10
of
344
Independent Component Analysis
 Neural Computing Surveys
, 2001
"... A common problem encountered in such disciplines as statistics, data analysis, signal processing, and neural network research, is nding a suitable representation of multivariate data. For computational and conceptual simplicity, such a representation is often sought as a linear transformation of the ..."
Abstract

Cited by 1492 (93 self)
 Add to MetaCart
A common problem encountered in such disciplines as statistics, data analysis, signal processing, and neural network research, is nding a suitable representation of multivariate data. For computational and conceptual simplicity, such a representation is often sought as a linear transformation of the original data. Wellknown linear transformation methods include, for example, principal component analysis, factor analysis, and projection pursuit. A recently developed linear transformation method is independent component analysis (ICA), in which the desired representation is the one that minimizes the statistical dependence of the components of the representation. Such a representation seems to capture the essential structure of the data in many applications. In this paper, we survey the existing theory and methods for ICA. 1
Data Clustering: A Review
 ACM COMPUTING SURVEYS
, 1999
"... Clustering is the unsupervised classification of patterns (observations, data items, or feature vectors) into groups (clusters). The clustering problem has been addressed in many contexts and by researchers in many disciplines; this reflects its broad appeal and usefulness as one of the steps in exp ..."
Abstract

Cited by 1284 (13 self)
 Add to MetaCart
Clustering is the unsupervised classification of patterns (observations, data items, or feature vectors) into groups (clusters). The clustering problem has been addressed in many contexts and by researchers in many disciplines; this reflects its broad appeal and usefulness as one of the steps in exploratory data analysis. However, clustering is a difficult problem combinatorially, and differences in assumptions and contexts in different communities has made the transfer of useful generic concepts and methodologies slow to occur. This paper presents an overview of pattern clustering methods from a statistical pattern recognition perspective, with a goal of providing useful advice and references to fundamental concepts accessible to the broad community of clustering practitioners. We present a taxonomy of clustering techniques, and identify crosscutting themes and recent advances. We also describe some important applications of clustering algorithms such as image segmentation, object recognition, and information retrieval.
Nonlinear component analysis as a kernel eigenvalue problem

, 1996
"... We describe a new method for performing a nonlinear form of Principal Component Analysis. By the use of integral operator kernel functions, we can efficiently compute principal components in highdimensional feature spaces, related to input space by some nonlinear map; for instance the space of all ..."
Abstract

Cited by 1048 (72 self)
 Add to MetaCart
We describe a new method for performing a nonlinear form of Principal Component Analysis. By the use of integral operator kernel functions, we can efficiently compute principal components in highdimensional feature spaces, related to input space by some nonlinear map; for instance the space of all possible 5pixel products in 16x16 images. We give the derivation of the method, along with a discussion of other techniques which can be made nonlinear with the kernel approach; and present first experimental results on nonlinear feature extraction for pattern recognition.
A Growing Neural Gas Network Learns Topologies
 Advances in Neural Information Processing Systems 7
, 1995
"... An incremental network model is introduced which is able to learn the important topological relations in a given set of input vectors by means of a simple Hebblike learning rule. In contrast to previous approaches like the "neural gas" method of Martinetz and Schulten (1991, 1994), this model has n ..."
Abstract

Cited by 285 (5 self)
 Add to MetaCart
An incremental network model is introduced which is able to learn the important topological relations in a given set of input vectors by means of a simple Hebblike learning rule. In contrast to previous approaches like the "neural gas" method of Martinetz and Schulten (1991, 1994), this model has no parameters which change over time and is able to continue learning, adding units and connections, until a performance criterion has been met. Applications of the model include vector quantization, clustering, and interpolation. 1 INTRODUCTION In unsupervised learning settings only input data is available but no information on the desired output. What can the goal of learning be in this situation? One possible objective is dimensionality reduction: finding a lowdimensional subspace of the input vector space containing most or all of the input data. Linear subspaces with this property can be computed directly by principal component analysis or iteratively with a number of network models (S...
Optimal Unsupervised Learning in a SingleLayer Linear Feedforward Neural Network
, 1989
"... A new approach to unsupervised learning in a singlelayer linear feedforward neural network is discussed. An optimality principle is proposed which is based upon preserving maximal information in the output units. An algorithm for unsupervised learning based upon a Hebbian learning rule, which achie ..."
Abstract

Cited by 218 (0 self)
 Add to MetaCart
A new approach to unsupervised learning in a singlelayer linear feedforward neural network is discussed. An optimality principle is proposed which is based upon preserving maximal information in the output units. An algorithm for unsupervised learning based upon a Hebbian learning rule, which achieves the desired optimality is presented, The algorithm finds the eigenvectors of the input correlation matrix, and it is proven to converge with probability one. An implementation which can train neural networks using only local "synaptic" modification rules is described. It is shown that the algorithm is closely related to algorithms in statistics (Factor Analysis and Principal Components Analysis) and neural networks (Selfsupervised Backpropagation, or the "encoder" problem). It thus provides an explanation of certain neural network behavior in terms of classical statistical techniques. Examples of the use of a linear network for solving image coding and texture segmentation problems are presented. Also, it is shown that the algorithm can be used to find "visual receptive fields" which are qualitatively similar to those found in primate retina and visual cortex.
Kernel principal component analysis
 ADVANCES IN KERNEL METHODS  SUPPORT VECTOR LEARNING
, 1999
"... A new method for performing a nonlinear form of Principal Component Analysis is proposed. By the use of integral operator kernel functions, one can efficiently compute principal components in highdimensional feature spaces, related to input space by some nonlinear map; for instance the space of all ..."
Abstract

Cited by 177 (8 self)
 Add to MetaCart
A new method for performing a nonlinear form of Principal Component Analysis is proposed. By the use of integral operator kernel functions, one can efficiently compute principal components in highdimensional feature spaces, related to input space by some nonlinear map; for instance the space of all possible dpixel products in images. We give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition.
Computational analysis of the role of the hippocampus in memory
 Hippocampus
, 1994
"... The authors draw together the results of a series of detailed computational studies and show how they are contributing to the development of a theory of hippocampal function. A new part of the theory introduced here is a quantitative analysis of how backprojections from the hippocampus to the neocor ..."
Abstract

Cited by 134 (14 self)
 Add to MetaCart
The authors draw together the results of a series of detailed computational studies and show how they are contributing to the development of a theory of hippocampal function. A new part of the theory introduced here is a quantitative analysis of how backprojections from the hippocampus to the neocortex could lead to the recall of recent memories. The theory is then compared with other theories of hippocampal function. First, what is computed by the hippocampus is considered. The hypothesis the authors advocate, on the basis of the effects of damage to the hippocampus and neuronal activity recorded in it, is that it is involved in the formation of new memories by acting as an intermediateterm buffer store for information about episodes, particularly for spatial, but probably also for some nonspatial, information. The authors analyze how the hippocampus could perform this function, by producing a computational theory of how it operates, based on neuroanatomical and neurophysiological information about the different neuronal systems contained within the hippocampus. Key hypotheses are that the CA3 pyramidal cells operate as a single autoassociation network to store new episodic information as it arrives via a number of specialized preprocessing stages from many association areas of the cerebral cortex, and that the dentate
Map Learning with Uninterpreted Sensors and Effectors
 Artificial Intelligence
, 1997
"... This paper presents a set of methods by which a learning agent can learn a sequence of increasingly abstract and powerful interfaces to control a robot whose sensorimotor apparatus and environment are initially unknown. The result of the learning is a rich hierarchical model of the robot's world (it ..."
Abstract

Cited by 129 (20 self)
 Add to MetaCart
This paper presents a set of methods by which a learning agent can learn a sequence of increasingly abstract and powerful interfaces to control a robot whose sensorimotor apparatus and environment are initially unknown. The result of the learning is a rich hierarchical model of the robot's world (its sensorimotor apparatus and environment). The learning methods rely on generic properties of the robot's world such as almosteverywhere smooth e ects of motor control signals on sensory features. At thelowest level of the hierarchy, the learning agent analyzes the e ects of its motor control signals in order to de ne a new set of control signals, one for each of the robot's degrees of freedom. It uses a generateandtest approach to de ne sensory features that capture important aspects of the environment. It uses linear regression to learn models that characterize contextdependent e ects of the control signals on the learned features. It uses these models to de ne highlevel control laws for nding and following paths de ned using constraints on the learned features. The agent abstracts these control laws, which interact with the continuous environment, to a nite set of actions that implement discrete state transitions. At this point, the agent has abstracted the robot's continuous world to a nitestate world and can use existing methods to learn its structure. The learning agent's methods are evaluated on several simulated robots with di erent sensorimotor systems and environments.
Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia
, 2003
"... The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and “executive ” functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the ..."
Abstract

Cited by 102 (14 self)
 Add to MetaCart
The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and “executive ” functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This paper presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, taskappropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia and amygdala, which together form an actor/critic architecture. The critic system learns which prefrontal representations are taskrelevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model’s performance compares favorably with standard backpropagationbased temporal learning mechanisms on the challenging 12AX working memory task, and other benchmark working memory tasks.
Dimension Reduction by Local Principal Component Analysis
, 1997
"... Reducing or eliminating statistical redundancy between the components of highdimensional vector data enables a lowerdimensional representation without significant loss of information. Recognizing the limitations of principal component analysis (PCA), researchers in the statistics and neural networ ..."
Abstract

Cited by 99 (0 self)
 Add to MetaCart
Reducing or eliminating statistical redundancy between the components of highdimensional vector data enables a lowerdimensional representation without significant loss of information. Recognizing the limitations of principal component analysis (PCA), researchers in the statistics and neural network communities have developed nonlinear extensions of PCA. This article develops a local linear approach to dimension reduction that provides accurate representations and is fast to compute. We exercise the algorithms on speech and image data, and compare performance with PCA and with neural network implementations of nonlinear PCA. We find that both nonlinear techniques can provide more accurate representations than PCA and show that the local linear techniques outperform neural network implementations.