Abstract

The error backpropagation learning algorithm (BP) is generally considered biologically implausible because it does not use locally available, activation-based variables. A version of BP that can be computed locally using bi-directional activation recirculation (Hinton & McClelland, 1988) instead of backpropagated error derivatives is more biologically plausible. This paper presents a generalized version of the recirculation algorithm (GeneRec), which overcomes several limitations of the earlier algorithm by using a generic recurrent network with sigmoidal units that can learn arbitrary input/output mappings. However, the contrastiveHebbian learning algorithm (CHL, a.k.a. DBM or mean field learning) also uses local variables to perform error-driven learning in a sigmoidal recurrent network. CHL was derived in a stochastic framework (the Boltzmann machine), but has been extended to the deterministic case in various ways, all of which rely on problematic approximationsand assumptions, le...

Citations

2304	Learning Internal Representations by Error Propagation - Rumelhart, Hinton, et al. - 1986
490	Neurons with graded response have collective computational properties like those of two-state neurons - Hopfield
364	A learning algorithm for Boltzmann Machines - Ackley, Hinton, et al. - 1985
339	Distributed hierarchical processing in the primate cerebral cortex - Felleman, Essen - 1991
290	Connectionist Learning Procedures - Hinton - 1990
245	Learning and relearning in Boltzmann machines - Hinton, Sejnowski - 1986
217	Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex - Bienenstock, Cooper, et al. - 1982
154	Learning distributed representations of concepts - Hinton - 1986
127	A mean field theory learning algorithm for neural networks - Peterson, Anderson - 1987
124	Generalization of backpropagation to recurrent neural networks - Pineda - 1987
108	Second-Order Methods for Learning: Between Steepest Descent and Newton's Method - Battiti, First- - 1992
75	Ljungberg T: Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task - Schultz, Apicella - 1993
68	A learning rule for asynchronous perceptrons with feedback in a combinatorial environment - Almeida - 1987
60	Long-term depression of excitatory synaptic transmission and its relationship to long-term potentiation. Trends in Neurosciences - Artola, Singer - 1993
57	A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature - ZIPSER, RA - 1988
56	Deterministic Boltzmann learning performs steepest descent in weight-space - Hinton - 1990
50	Different voltagedependent thresholds for inducing long-term depression and long-term potentiation in slices of rat visual cortex - Artola, Brocher, et al. - 1990
49	Local synaptic learning rules suffice to maximize mutual information in a linear network - Linsker - 1992
42	The recent excitement about neural networks - Crick - 1989
41	A mechanism for the Hebb and the anti-Hebb processes underlying learning and memory - Lisman - 1989
40	Numerical Recipies: The art of Scientific Computing - Press, Teukolsky, et al. - 1992
39	Dynamics and architecture for neural computation - Pineda - 1988
38	Contrastive Hebbian learning in the continuous Hopfield model - Movellan - 1990
34	Explorations of the mean field theory learning algorithm." Neural Networks - Peterson, Hartrnan - 1989
34	JL: Parallel Distributed Processing. Volume 1 and 2 - DE, McClelland - 1986
32	Topography of pyramidal neuron intrinsic connections in macaque monkey prefrontal cortex (areas 9 - Levitt, Lewis, et al. - 1993
31	Learning representations by recirculation - Hinton, McClelland - 1988
29	A more biologically plausible learning rule for neural networks - Mazzoni, Andersen, et al. - 1991
25	The LEABRA model of neural interactions and learning in the neocortex - O’Reilly
25	Evoked-potential correlates of stimulus uncertainty - Sutton, Braren, et al. - 1965
24	The function of dream sleep - Crick, Mitchison - 1983
19	The CaM kinase II hypothesis for the storage of synaptic memory. Trends Neurosci 17 - Lisman - 1994
19	Generalization of back propagation to recurrent and higher order neural networks - Pineda - 1987
17	The limitations of deterministic Boltzmann machine learning - Galland - 1993
16	Synaptic plasticity - Bear, Malenka - 1994
16	A biologically supported error-correcting learning rule - Hancock, Smith, et al. - 1991
15	The interaction of nature and nurture in development: A parallel distributed processing perspective - McClelland - 1994
13	The spread of Na ÷ spikes determines the pattern of dendritic Ca 2÷ entry into hippocampal neurons. Nature (Lond - Jaffe, Johnston, et al. - 1992
11	NMDA receptors: Their role in long-term potentiation. Trends Neurosci - Collingridge, Bliss - 1987
10	Deterministic Boltzmann learning in networks with asymmetric connectivity - Galland, Hinton
10	NMDAreceptor-dependent synaptic plasticity: multiple forms and mechanisms. Trends Neurosci - Malenka, Nicoll - 1993
7	Long-term synaptic depression in the mammalian brain - Linden - 1994
5	Experiments on discovering high order features with mean field modules (Technical Report CRG-TR-90-3 - Galland - 1990
4	Intracellular injection of Ca2+ chelators blocks induction of long-term depression in rat visual cortex - Artola, A, et al. - 1992
3	NMDA receptors - their role in long-term potentiation - Bliss - 1987
3	Electrophysiology of cognition. In F. Plum (Ed.), Handbook of physiology: Section 1. The nervous system - Hillyard, Picton - 1987
3	Ca 2+ entry via postsynaptic voltagesensitive Ca 2+ channels can transiently potentiate excitatory synaptic transmission in the hippocampus - Kullmann, Perkel, et al. - 1992
3	Neural models of classical conditioning: A theoretical viewpoint - Tesauro - 1990
2	Parallel DistributedProcessing volumes 1 and 2 - McClelland, Rumelhart - 1986
1	Induction of LTP in the hippocampus needs synaptic activation of glutamate metabotropic receptors - Bashir, Bortolotto - 1993