Results 1  10
of
28
Connectionist Learning Procedures
 ARTIFICIAL INTELLIGENCE
, 1989
"... A major goal of research on networks of neuronlike processing units is to discover efficient learning procedures that allow these networks to construct complex internal representations of their environment. The learning procedures must be capable of modifying the connection strengths in such a way ..."
Abstract

Cited by 339 (6 self)
 Add to MetaCart
A major goal of research on networks of neuronlike processing units is to discover efficient learning procedures that allow these networks to construct complex internal representations of their environment. The learning procedures must be capable of modifying the connection strengths in such a way that internal units which are not part of the input or output come to represent important features of the task domain. Several interesting gradientdescent procedures have recently been discovered. Each connection computes the derivative, with respect to the connection strength, of a global measure of the error in the performance of the network. The strength is then adjusted in the direction that decreases the error. These relatively simple, gradientdescent learning procedures work well for small tasks and the new challenge is to find ways of improving their convergence rate and their generalization abilities so that they can be applied to larger, more realistic tasks.
Learning continuous probability distributions with symmetric diffusion networks
 Cognitive Science
, 1993
"... in this article we present symmetric diffusion networks, a family of networks that instantiate the principles of continuous, stochastic, adaptive and interactive propagation of information. Using methods of Markovlon diffusion theory, we formalize the activation dynamics of these networks and then ..."
Abstract

Cited by 32 (7 self)
 Add to MetaCart
in this article we present symmetric diffusion networks, a family of networks that instantiate the principles of continuous, stochastic, adaptive and interactive propagation of information. Using methods of Markovlon diffusion theory, we formalize the activation dynamics of these networks and then show that they can be trained to reproduce entire muitivariote probability distributions an their outputs using the contrastive Hebbian learning rule (CHL).,We show that CHL performs gradient descent on an error function that captures differences between desired and obtolned continuous multivoriate probability distributions. This allows the learning algorithm to go beyond expected values of output units and to approximate complete probability distributions on continuous muitivariote activation spaces. We argue that learning continuous distributions is an important task underlying a variety of reallife situations that were beyond the scope of previous connectionist networks. Deterministic networks, like back propagation, cannot ieorn this task because they ore limited to learning average values of independent output units. Previous stochastic connectionist networks could learn probobility distributions but they were limited to discrete variables. Simulations show that symmetric diffusion networks can be trained with the CHL rule to opproximate discrete and continuous probability distributions of various types. 1.
Learning continuous attractors in recurrent networks
 Advances in Neural Information Processing Systems
, 1998
"... One approach toinvariant object recognition employs a recurrent neural network as an associative memory. In the standard depiction of the network's state space, memories of objects are stored as attractive xed points of the dynamics. I argue for a modi cation of this picture: if an object has a cont ..."
Abstract

Cited by 29 (5 self)
 Add to MetaCart
One approach toinvariant object recognition employs a recurrent neural network as an associative memory. In the standard depiction of the network's state space, memories of objects are stored as attractive xed points of the dynamics. I argue for a modi cation of this picture: if an object has a continuous family of instantiations, it should be represented by a continuous attractor. This idea is illustrated with a network that learns to complete patterns. To perform the task of lling in missing information, the network develops a continuous attractor that models the manifold from which the patterns are drawn. From a statistical viewpoint, the pattern completion task allows a formulation of unsupervised learning in terms of regression rather than density estimation. A classic approach toinvariant object recognition is to use a recurrent neural network as an associative memory[1]. In spite of the intuitive appeal and biological plausibility of this approach, it has largely been abandoned in practical applications.
GAL: Networks that grow when they learn and shrink when they forget
 INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE
, 1991
"... Learning when limited to modification of some parameters has a limited scope; the capability to modify the system structure is also needed to get a wider range of the learnable. In the case of artificial neural networks, learning by iterative adjustment of synaptic weights can only succeed if t ..."
Abstract

Cited by 21 (4 self)
 Add to MetaCart
Learning when limited to modification of some parameters has a limited scope; the capability to modify the system structure is also needed to get a wider range of the learnable. In the case of artificial neural networks, learning by iterative adjustment of synaptic weights can only succeed if the network designer predefines an appropriate network structure, i.e., number of hidden layers, units, and the size and shape of their receptive and projective fields. This paper advocates the view that the network structure should not, as usually done, be determined by trialanderror but should be computed by the learning algorithm. Incremental learning algorithms can modify the network structure by addition and/or removal of units and/or links. A survey of current connectionist literature is given on this line of thought. "Grow and Learn" (GAL) is a new algorithm that learns an association at oneshot due to being incremental and using a local representation. During the socalled...
Memory Maintenance Via Neuronal Regulation
 Neural Computation
, 1998
"... Since their conception half a century ago Hebbian cell assemblies have become a basic term in the Neurosciences, and the idea that learning takes place through synaptic modifications has been accepted as a fundamental paradigm. As synapses undergo continuous metabolic turnover, adopting the stance t ..."
Abstract

Cited by 19 (5 self)
 Add to MetaCart
Since their conception half a century ago Hebbian cell assemblies have become a basic term in the Neurosciences, and the idea that learning takes place through synaptic modifications has been accepted as a fundamental paradigm. As synapses undergo continuous metabolic turnover, adopting the stance that memories are engraved in the synaptic matrix raises a fundamental problem: How can memories be maintained for very long time periods? We present a novel solution to this longstanding question, that is based on biological evidence of neuronal regulation mechanisms that act to maintain neuronal activity. Our mechanism is developed within the framework of a neural model of associative memory. It is operative in conjunction with random activation of the memory system, and is able to counterbalance degradation of synaptic weights, and to normalize the basins of attraction of all memories. Over long time periods, when the variance of the degradation process becomes important, the memory syste...
Learning symmetry groups with hidden units: Beyond the perceptron
 Physica
, 1986
"... Learning to recognize mirror, rotational and translational symmetries is a difficult problem for massivelyparallel network models. These symmetries cannot be learned by firstorder perceptrons or Hopfield networks, which have no means for incorporating additional adaptive units that are hidden from ..."
Abstract

Cited by 17 (5 self)
 Add to MetaCart
Learning to recognize mirror, rotational and translational symmetries is a difficult problem for massivelyparallel network models. These symmetries cannot be learned by firstorder perceptrons or Hopfield networks, which have no means for incorporating additional adaptive units that are hidden from the input and output layers. We demonstrate that the Boltzmann learning algorithm is capable of finding sets of weights which turn hidden units into useful higherorder feature detectors capable of solving symmetry problems. 1.
On the Impact of Forgetting on Learning Machines
 Journal of the ACM
, 1993
"... this paper contributes toward the goal of understanding how a computer can be programmed to learn by isolating features of incremental learning algorithms that theoretically enhance their learning potential. In particular, we examine the effects of imposing a limit on the amount of information that ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
this paper contributes toward the goal of understanding how a computer can be programmed to learn by isolating features of incremental learning algorithms that theoretically enhance their learning potential. In particular, we examine the effects of imposing a limit on the amount of information that learning algorithm can hold in its memory as it attempts to This work was facilitated by an international agreement under NSF Grant 9119540.
Catastrophic forgetting and the pseudorehearsal solution in Hopfieldtype networks
 Connection Science
, 1998
"... Pseudorehearsal is a mechanism proposed by Robins which alleviates catastrophic forgetting in multilayer perceptron networks. In this paper, we extend the exploration of pseudorehearsal to a Hop ® eldtype net. The same general principles apply: old information can be rehearsed if it is available, ..."
Abstract

Cited by 10 (4 self)
 Add to MetaCart
Pseudorehearsal is a mechanism proposed by Robins which alleviates catastrophic forgetting in multilayer perceptron networks. In this paper, we extend the exploration of pseudorehearsal to a Hop ® eldtype net. The same general principles apply: old information can be rehearsed if it is available, and if it is not available, then generating and rehearsing approximations of old information that `map ’ the behaviour of the network can also be eŒective at preser ving the actual old information itself. The details of the pseudorehearsal mechanism, however, bene ® t from being adapted to the dynamics of Hop ® eld nets so as to exploit the extra attractors created in state space during learning. These attractors are usually described as `spurious ’ or `crosstalk’, and regarded as undesirable, interfering with the retention of the trained population items. Our simulations have shown that, in another sense, such attractors can in fact be useful in preser ving the learned population. In general terms, a solution to the catastrophic forgetting problem enables the ongoing or sequential learning of information in arti ® cial neural networks, and consequently also provides a framework for the modelling of lifelong learning/developmental eŒects in cognition.
Attractors In Recurrent Behavior Networks
, 1997
"... If behavior networks, which use spreading activation to select actions, are analogous to connectionist methods of pattern recognition, then recurrent behavior networks, which use energy minimization, are analogous to Hopfield networks. Hopfield networks memorize patterns by making them attractors. S ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
If behavior networks, which use spreading activation to select actions, are analogous to connectionist methods of pattern recognition, then recurrent behavior networks, which use energy minimization, are analogous to Hopfield networks. Hopfield networks memorize patterns by making them attractors. Similarly, each behavior of a recurrent behavior network should be an attractor of the network, to inhibit fruitless, repeated switching between different behaviors in response to small changes in the environment and in motivations. I overcome two major objections to this view, and demonstrate that the performance in a test domain of the Do the Right Thing recurrent behavior network is improved by redesigning it to create desirable attractors and basins of attraction. I further show that this performance increase is correlated with an increase in persistence and a decrease in undesirable behaviorswitching. On a more general level, this work encourages the study of action selection as a dynam...
Amplifying phenomenal information: Toward a fundamental theory of consciousness
 Journal of Consciousness Studies
, 2002
"... Abstract: Fundamental approaches bypass the problem of getting consciousness from nonconscious components by positing that consciousness is a universal primitive. For example, the double aspect theory of information holds that information has a phenomenal aspect. How then do you get from phenomenal ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
Abstract: Fundamental approaches bypass the problem of getting consciousness from nonconscious components by positing that consciousness is a universal primitive. For example, the double aspect theory of information holds that information has a phenomenal aspect. How then do you get from phenomenal information to human consciousness? This paper proposes that an entity is conscious to the extent it amplifies information, first by trapping and integrating it through closure, and second by maintaining dynamics at the edge of chaos through simultaneous processes of divergence and convergence. The origin of life through autocatalytic closure, and the origin of an interconnected worldview through conceptual closure, induced phase transitions in the degree to which information, and thus consciousness, is locally amplified. Divergence and convergence of cognitive information may involve phenomena observed in light e.g. focusing, interference, and resonance. By making information flow inward biased, closure shields us from external consciousness; thus the paucity of consciousness may be an illusion.