Results 1  10
of
38
Learning Invariance From Transformation Sequences
, 1991
"... Introduction How can we consistently recognize objects when changes in the viewing angle, eye position, distance, size, orientation, relative position, or deformations of the object itself (e.g., of a newspaper or a gymnast) can change their retinal projections so significantly? The visual system m ..."
Abstract

Cited by 240 (2 self)
 Add to MetaCart
Introduction How can we consistently recognize objects when changes in the viewing angle, eye position, distance, size, orientation, relative position, or deformations of the object itself (e.g., of a newspaper or a gymnast) can change their retinal projections so significantly? The visual system must contain knowledge about such transformations in order to be able to generalize correctly. Part of this knowledge is probably determined genetically, but it is also likely that the visual system learns from its sensory experience, which contains plenty of examples of such transformations. Electrophysiological experiments suggest that the invariance properties of perception may be due to the receptive field characteristics of individual cells in the visual system. Complex cells in the primary visual cortex exhibit approximate invariance to position within a limited range (Hubel and Wiesel 1962), while cells in higher visual areas in the temporal cortex show more complex forms of invariance
Task Decomposition Through Competition in a Modular Connectionist Architecture
 COGNITIVE SCIENCE
, 1990
"... A novel modular connectionist architecture is presented in which the networks composing the architecture compete to learn the training patterns. As a result of the competition, different networks learn different training patterns and, thus, learn to compute different functions. The architecture pe ..."
Abstract

Cited by 181 (5 self)
 Add to MetaCart
A novel modular connectionist architecture is presented in which the networks composing the architecture compete to learn the training patterns. As a result of the competition, different networks learn different training patterns and, thus, learn to compute different functions. The architecture performs task decomposition in the sense that it learns to partition a task into two or more functionally independent vii tasks and allocates distinct networks to learn each task. In addition, the architecture tends to allocate to each task the network whose topology is most appropriate to that task, and tends to allocate the same network to similar tasks and distinct networks to dissimilar tasks. Furthermore, it can be easily modified so as to...
Non Linear Neurons in the Low Noise Limit: A Factorial Code Maximizes Information Transfer
, 1994
"... We investigate the consequences of maximizing information transfer in a simple neural network (one input layer, one output layer), focussing on the case of non linear transfer functions. We assume that both receptive fields (synaptic efficacies) and transfer functions can be adapted to the environm ..."
Abstract

Cited by 141 (18 self)
 Add to MetaCart
We investigate the consequences of maximizing information transfer in a simple neural network (one input layer, one output layer), focussing on the case of non linear transfer functions. We assume that both receptive fields (synaptic efficacies) and transfer functions can be adapted to the environment. The main result is that, for bounded and invertible transfer functions, in the case of a vanishing additive output noise, and no input noise, maximization of information (Linsker'sinfomax principle) leads to a factorial code  hence to the same solution as required by the redundancy reduction principle of Barlow. We show also that this result is valid for linear, more generally unbounded, transfer functions, provided optimization is performed under an additive constraint, that is which can be written as a sum of terms, each one being specific to one output neuron. Finally we study the effect of a non zero input noise. We find that, at first order in the input noise, assumed to be small ...
Adaptive OnLine Learning Algorithms for Blind Separation  Maximum Entropy and Minimum Mutual Information
 Neural Computation
, 1997
"... There are two major approaches for blind separation: Maximum Entropy (ME) and Minimum Mutual Information (MMI). Both can be implemented by the stochastic gradient descent method for obtaining the demixing matrix. The MI is the contrast function for blind separation while the entropy is not. To just ..."
Abstract

Cited by 110 (16 self)
 Add to MetaCart
There are two major approaches for blind separation: Maximum Entropy (ME) and Minimum Mutual Information (MMI). Both can be implemented by the stochastic gradient descent method for obtaining the demixing matrix. The MI is the contrast function for blind separation while the entropy is not. To justify the ME, the relation between ME and MMI is firstly elucidated by calculating the first derivative of the entropy and proving that 1) the the meansubtraction is necessary in applying the ME and 2) at the solution points determined by the MI the ME will not update the demixing matrix in the directions of increasing the crosstalking. Secondly, the natural gradient instead of the ordinary gradient is introduced to obtain efficient algorithms, because the parameter space is a Riemannian space consisting of matrices. The mutual information is calculated by applying the GramCharlier expansion to approximate probability density functions of the outputs. Finally, we propose an efficient learn...
Learning in Linear Neural Networks: a Survey
 IEEE Transactions on neural networks
, 1995
"... Networks of linear units are the simplest kind of networks, where the basic questions related to learning, generalization, and selforganisation can sometimes be answered analytically. We survey most of the known results on linear networks, including: (1) backpropagation learning and the structure ..."
Abstract

Cited by 56 (4 self)
 Add to MetaCart
Networks of linear units are the simplest kind of networks, where the basic questions related to learning, generalization, and selforganisation can sometimes be answered analytically. We survey most of the known results on linear networks, including: (1) backpropagation learning and the structure of the error function landscape; (2) the temporal evolution of generalization; (3) unsupervised learning algorithms and their properties. The connections to classical statistical ideas, such as principal component analysis (PCA), are emphasized as well as several simple but challenging open questions. A few new results are also spread across the paper, including an analysis of the effect of noise on backpropagation networks and a unified view of all unsupervised algorithms. Keywords linear networks, supervised and unsupervised learning, Hebbian learning, principal components, generalization, local minima, selforganisation I. Introduction This paper addresses the problems of supervise...
Recurrent Neural Networks for Blind Separation of Sources
 INTERNATIONAL SYMPOSIUM ON NONLINEAR THEORY AND ITS APPLICATIONS, NOLTA'95, LAS VEGAS, DECEMBER, 1995.
, 1995
"... In this paper, fully connected recurrent neural networks are investigated for blind separation of sources. For these networks, a new class of unsupervised online learning algorithms are proposed. These algorithms are the generalization of the Hebbian/antiHebbian rule. They are not only biologicall ..."
Abstract

Cited by 47 (23 self)
 Add to MetaCart
In this paper, fully connected recurrent neural networks are investigated for blind separation of sources. For these networks, a new class of unsupervised online learning algorithms are proposed. These algorithms are the generalization of the Hebbian/antiHebbian rule. They are not only biologically plausible but also theoretically sound. An important property of these algorithms is that the performance of the networks is independent of the mixing matrix and the scaling factor of the input sources. This property is verified by analyses and simulations.
Visual Adaptation as Optimal Information Transmission
 Vision Research
, 1999
"... We propose that visual adaptation in orientation, spatial frequency, and motion can be understood from the perspective of optimal information transmission. The essence of the proposal is that neural response properties at the system level should be adjusted to the changing statistics of the input so ..."
Abstract

Cited by 30 (1 self)
 Add to MetaCart
We propose that visual adaptation in orientation, spatial frequency, and motion can be understood from the perspective of optimal information transmission. The essence of the proposal is that neural response properties at the system level should be adjusted to the changing statistics of the input so as to maximize information transmission. We show that this principle accounts for several welldocumented psychophysical phenomena, including the tilt aftereffect, change in contrast sensitivity and postadaptation changes in orientation discrimination. Adaptation can also be considered on a longer time scale, in the context of tailoring response properties to natural scene statistics. From the anisotropic distribution of power in natural scenes, the proposal also predicts differences in the contrast sensitivity function across spatial frequency and orientation, including the oblique effect. 1999 Elsevier Science Ltd. All rights reserved. Keywords: Adaptation; Aftereffects; Signaltonoise ratio; Sensitivity; Natural scenes www.elsevier.com/locate/visres 1.
A functional angle on some aftereffects in cortical vision
, 2000
"... The question of how our brains and those of other animals code sensory information is of fundamental importance to neuroscience research. Visual illusions offer valuable insight into the mechanisms of perceptual coding. One such illusion, the tilt aftereffect (TAE), has been studied extensively sin ..."
Abstract

Cited by 21 (1 self)
 Add to MetaCart
The question of how our brains and those of other animals code sensory information is of fundamental importance to neuroscience research. Visual illusions offer valuable insight into the mechanisms of perceptual coding. One such illusion, the tilt aftereffect (TAE), has been studied extensively since the 1930s, yet a full explanation of the effect has remained elusive. Here, we put forward an explanation of the TAE in terms of a functional role for adaptation in the visual cortex. The proposed model accounts not only for the phenomenology of the TAE, but also for spatial interactions in perceived tilt and the e¡ects of adaptation on the perception of direction of motion and colour. We discuss the implications of the model for understanding the effects of adaptation and surround stimulation on the response properties of cortical neurons.
Weighted Linear Cue Combination with Possibly Correlated Error
 AMERICAN DOCUMENTATION
, 2003
"... We test hypotheses concerning human cue combination in a slant estimation task. Observers ..."
Abstract

Cited by 20 (9 self)
 Add to MetaCart
We test hypotheses concerning human cue combination in a slant estimation task. Observers