Results 1  10
of
29
Biologically Plausible Errordriven Learning using Local Activation Differences: The Generalized Recirculation Algorithm
 NEURAL COMPUTATION
, 1996
"... The error backpropagation learning algorithm (BP) is generally considered biologically implausible because it does not use locally available, activationbased variables. A version of BP that can be computed locally using bidirectional activation recirculation (Hinton & McClelland, 1988) instead of ..."
Abstract

Cited by 94 (10 self)
 Add to MetaCart
The error backpropagation learning algorithm (BP) is generally considered biologically implausible because it does not use locally available, activationbased variables. A version of BP that can be computed locally using bidirectional activation recirculation (Hinton & McClelland, 1988) instead of backpropagated error derivatives is more biologically plausible. This paper presents a generalized version of the recirculation algorithm (GeneRec), which overcomes several limitations of the earlier algorithm by using a generic recurrent network with sigmoidal units that can learn arbitrary input/output mappings. However, the contrastiveHebbian learning algorithm (CHL, a.k.a. DBM or mean field learning) also uses local variables to perform errordriven learning in a sigmoidal recurrent network. CHL was derived in a stochastic framework (the Boltzmann machine), but has been extended to the deterministic case in various ways, all of which rely on problematic approximationsand assumptions, le...
Conjunctive Representations in Learning and Memory: Principles of Cortical and Hippocampal Function
 PSYCHOLOGICAL REVIEW
, 2001
"... We present a theoretical framework for understanding the roles of the hippocampus and neocortex in learning and memory. This framework incorporates a theme found in many theories of hippocampal function, that the hippocampus is responsible for developing conjunctive representations binding together ..."
Abstract

Cited by 91 (11 self)
 Add to MetaCart
We present a theoretical framework for understanding the roles of the hippocampus and neocortex in learning and memory. This framework incorporates a theme found in many theories of hippocampal function, that the hippocampus is responsible for developing conjunctive representations binding together stimulus elements into a unitary rep resentation that can later be recalled from partial input cues. This idea appears problematic, however, because it is contradicted by the fact that hippocampally lesioned rats can learn nonlinear discrimination problems that require conjunctive representations. Our framework accommodates this finding by establishing a principled division of labor between the cortex and hippocampus, where the cortex is responsible for slow learning that integrates over multiple experiences to extract generalities, while the hippocampus performs rapid learning of the arbitrary contents of individual experiences. This framework shows that nonlinear discrimination problems are not good tests of hippocampal function, and suggests that tasks involving rapid, incidental conjunctive learning are better. We implement this framework in a computational neural network model, and show that it can account for a wide range of data in animal learning, thus validating our theoretical ideas, and providing a number of insights and predictions about these learning phenomena.
Six Principles for BiologicallyBased Computational Models of Cortical Cognition
 TRENDS IN COGNITIVE SCIENCES
, 1998
"... This paper describes and motivates six principles for computational cognitive neuroscience models: biological realism, distributed representations, inhibitory competition, bidirectional activation propagation, errordriven task learning, and Hebbian model learning. Although these principles are suppo ..."
Abstract

Cited by 52 (14 self)
 Add to MetaCart
This paper describes and motivates six principles for computational cognitive neuroscience models: biological realism, distributed representations, inhibitory competition, bidirectional activation propagation, errordriven task learning, and Hebbian model learning. Although these principles are supported by a number of cognitive, computational, and biological motivations, the prototypical neural network model (a feedforward backpropagation network) incorporates only two of them, and no widely used model incorporates all of them. This paper argues that these principles should be integrated into a coherent overall framework, and discusses some potential synergies and conflicts in doing so.
From the lexicon to expectations about kinds: a role for associative learning
 Psychological Review
, 2005
"... In the novel noun generalization task, 2 1/2yearold children display generalized expectations about how solid and nonsolid things are named, extending names for neverbeforeencountered solids by shape and for neverbeforeencountered nonsolids by material.This distinction between solids and nonso ..."
Abstract

Cited by 49 (18 self)
 Add to MetaCart
In the novel noun generalization task, 2 1/2yearold children display generalized expectations about how solid and nonsolid things are named, extending names for neverbeforeencountered solids by shape and for neverbeforeencountered nonsolids by material.This distinction between solids and nonsolids has been interpreted in terms of an ontological distinction between objects and substances.Nine simulations and behavioral experiments tested the hypothesis that these expectations arise from the correlations characterizing early learned noun categories.In the simulation studies, connectionist networks were trained on noun vocabularies modeled after those of children.These networks formed generalized expectations about solids and nonsolids that match children’s performances in the novel noun generalization task in the very different languages of English and Japanese.The simulations also generate new predictions supported by new experiments with children.Implications are discussed in terms of children’s development of distinctions between kinds of categories and in terms of the nature of this knowledge. Concepts are hypothetical constructs, theoretical devices hypothesized to explain data, what people do, and what people say. The question of whether a particular theory can explain children’s concepts is therefore semantically strange because strictly speaking this question asks about an explanation of an explanation.We begin with this reminder because the goal of the research reported here is to understand the role of associative processes in children’s systematic attention to the shape of solid things and to the material of nonsolid things in the task of forming new lexical categories. These attentional biases have been interpreted in terms of children’s concepts about the ontological kinds of object and substance
Generalization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian Learning
 Neural Computation
, 2001
"... Computational models in cognitive neuroscience should ideally use biological properties and powerful computational principles to produce behavior consistent with psychological findings. Errordriven backpropagation is computationally powerful, and has proven useful for modeling a range of psycholo ..."
Abstract

Cited by 45 (5 self)
 Add to MetaCart
Computational models in cognitive neuroscience should ideally use biological properties and powerful computational principles to produce behavior consistent with psychological findings. Errordriven backpropagation is computationally powerful, and has proven useful for modeling a range of psychological data, but is not biologically plausible. Several approaches to implementing backpropagation in a biologically plausible fashion converge on the idea of using bidirectional activation propagation in interactive networks to convey error signals. This paper demonstrates two main points about these errordriven interactive networks: (a) they generalize poorly due to attractor dynamics that interfere with the network's ability to systematically produce novel combinatorial representations in response to novel inputs; and (b) this generalization problem can be remedied by adding two widely used mechanistic principles, inhibitory competition and Hebbian learning, that can be independent...
A New Learning Algorithm for Mean Field Boltzmann Machines
, 2002
"... We present a new learning algorithm for Mean Field Boltzmann Machines based on the contrastive divergence optimization criterion. In addition to minimizing the divergence between the data distribution and the equilibrium distribution that the network believes in, we maximize the divergence betwe ..."
Abstract

Cited by 33 (8 self)
 Add to MetaCart
We present a new learning algorithm for Mean Field Boltzmann Machines based on the contrastive divergence optimization criterion. In addition to minimizing the divergence between the data distribution and the equilibrium distribution that the network believes in, we maximize the divergence between onestep reconstructions of the data and the equilibrium distribution. This eliminates the need to estimate equilibrium statistics, so we do not need to approximate the multimodal probablility distribution of the free network with the unimodal mean field distribution. We test the learning algorithm on the classification of digits. A New Learning Algorithm for Mean Field Boltzmann Machines Max Welling G.E. Hinton Gatsby Unit 1 Boltzmann Machines The stochastic Boltzmann machine (BM) is a probabilistic neural network of symmetrically connected binary units taking values f0; 1g (Ackley, Hinton & Sejnowski, 1985). The variant used for unsupervised learning consists of a set of visi...
Learning continuous probability distributions with symmetric diffusion networks
 Cognitive Science
, 1993
"... in this article we present symmetric diffusion networks, a family of networks that instantiate the principles of continuous, stochastic, adaptive and interactive propagation of information. Using methods of Markovlon diffusion theory, we formalize the activation dynamics of these networks and then ..."
Abstract

Cited by 32 (7 self)
 Add to MetaCart
in this article we present symmetric diffusion networks, a family of networks that instantiate the principles of continuous, stochastic, adaptive and interactive propagation of information. Using methods of Markovlon diffusion theory, we formalize the activation dynamics of these networks and then show that they can be trained to reproduce entire muitivariote probability distributions an their outputs using the contrastive Hebbian learning rule (CHL).,We show that CHL performs gradient descent on an error function that captures differences between desired and obtolned continuous multivoriate probability distributions. This allows the learning algorithm to go beyond expected values of output units and to approximate complete probability distributions on continuous muitivariote activation spaces. We argue that learning continuous distributions is an important task underlying a variety of reallife situations that were beyond the scope of previous connectionist networks. Deterministic networks, like back propagation, cannot ieorn this task because they ore limited to learning average values of independent output units. Previous stochastic connectionist networks could learn probobility distributions but they were limited to discrete variables. Simulations show that symmetric diffusion networks can be trained with the CHL rule to opproximate discrete and continuous probability distributions of various types. 1.
The dynamics of perceptual learning: an incremental reweighting model
 PSYCHOLOGICAL REVIEW
, 2005
"... The mechanisms of perceptual learning are analyzed theoretically, probed in an orientationdiscrimination experiment involving a novel nonstationary context manipulation, and instantiated in a detailed computational model. Two hypotheses are examined: modification of early cortical representations ve ..."
Abstract

Cited by 21 (2 self)
 Add to MetaCart
The mechanisms of perceptual learning are analyzed theoretically, probed in an orientationdiscrimination experiment involving a novel nonstationary context manipulation, and instantiated in a detailed computational model. Two hypotheses are examined: modification of early cortical representations versus taskspecific selective reweighting. Representation modification seems neither functionally necessary nor implied by the available psychophysical and physiological evidence. Computer simulations and mathematical analyses demonstrate the functional and empirical adequacy of selective reweighting as a perceptual learning mechanism. The stimulus images are processed by standard orientation and frequencytuned representational units, divisively normalized. Learning occurs only in the “readout” connections to a decision unit; the stimulus representations never change. An incremental Hebbian rule tracks the taskdependent predictive value of each unit, thereby improving the signaltonoise ratio of their weighted combination. Each abrupt change in the environmental statistics induces a switch cost in the learning curves as the system temporarily works with suboptimal weights.
A knowledgeresonance (KRES) model of category learning
 Psychonomic Bulletin & Review
, 2003
"... In this article we present a connectionist model of category learning that takes into account the prior knowledge that people bring to many new learning situations. This model, which we call the KnowledgeResonance Model or KRES, employs a recurrent network with bidirectional connections which are u ..."
Abstract

Cited by 19 (9 self)
 Add to MetaCart
In this article we present a connectionist model of category learning that takes into account the prior knowledge that people bring to many new learning situations. This model, which we call the KnowledgeResonance Model or KRES, employs a recurrent network with bidirectional connections which are updated according to a contrastiveHebbian learning rule. When prior knowledge is incorporated into a KRES network, the KRES activation dynamics and learning procedure accounts for a range of empirical results regarding the effects prior knowledge on category learning, including the accelerated learning that occurs in the presence of knowledge, the reinterpretation of features in light error correcting feedback, and the unlearning of prior knowledge which is inappropriate for a particular category.
How inhibitory oscillations can train neural networks and punish competitors
 Neural Computation
, 2006
"... The first two authors contributed equally to this research Revised manuscript, submitted to Neural Computation ..."
Abstract

Cited by 12 (5 self)
 Add to MetaCart
The first two authors contributed equally to this research Revised manuscript, submitted to Neural Computation