Stability and encoding properties of two-layer nonlinear feedback neural networks are examined. Bidirectionality, forward and backard information flow, is introduced in neural nets to produce two-way associative search for stored associations (A, B, ). Passing information through M gives one direction; passing it through its transpose M r gives the other. A bidirectional associative memory. (BAM) behaves as a hetero- associative content addressable memory (CAM), storing and recalling the vector pairs (A1, Bi),-..,(Am Bin) , where .4 {0,1}"and B We prove that every n-by-p matrix M is a bidirectionally stable heteroas- sociative CAM for both binary/bipolar and continuous neurons a, and hi. When the BAM neurons are activated, the network quickly evolves to a stable state of two-pattern reverberation, or resonance. The stable reverberation corresponds to a system energy local minimum. Heteroassociafive inlormation is encoded iu a BAM by summing correlation matrices. The BAM storage capact .ty for reliable recall is roughly m < niin(n, p). No more heteroassociafive pairs can be 'reliably stored and recalled than the lesser of the dimensions of the pattern spaces (0,1 }"and 0,1 } P. The Appendix shos that it is better on average to use bipolar {- 1,i} coding than binary. {0,1 } coding of heteroassociative pairs (.4, B,). BAM encoding and decoding are combined in the adaptive BAM, which extends global bidirectional stabflit), to realtime unsupervised learning. Temporal patterns (AE,--., A,,) are represented as ordered lists of binary/bipolar vectors and stored in a temporal associative memory (TAM) n-by- matrix M as a limit cycle of the dynamical system. Forward recall proceeds through M, backward recall through M r . Temporal patterns are stored by summing contiguous bipolar...

We describe a distributed model of information processing and memory and apply it to the representation of general and specific information. The model consists of a large number of simple processing elements which send excitatory and inhibitory signals to each other via modifiable connections. Information processing is thought of as the process whereby patterns of activation are formed over the units in the model through their excitatory and inhibitory interactions. The memory trace of a processing event is the change or increment to the strengths of the interconnections that results from the processing event. The traces of separate events are superimposed on each other in the values of the connection strengths that result from the entire set of traces stored in the memory. The model is applied to a number of findings related to the question of whether we store abstract representations or an enumeration of specific experiences in memory. The model simulates the results of a number of important experiments which have been taken as evidence for the enumeration of specific experiences. At the same time, it shows how the functional equivalent of abstract representations—prototypes, logogens

Abstract not found

We describe a testbed which can be used to investigate different codings for automatic face recognition. An eigenface coding of shape-free faces, based on manually coded landmarks was found to be more effective than the corresponding coding of correctly shaped faces. The advantage for shape-free faces was found to reflect and depend upon high-quality representation of the facial variation by means of an ensemble of shape-free faces. Configuration also proved an effective method of recognition, with rankings given to incorrect matches relatively uncorrelated with those from shape-free faces. Both sets of information combine to improve significantly the performance of either system. Manipulation within the coding to emphasize distinctive features of the faces, by caricaturing, allowed further increases in performance; this effect was noticeably larger when the independent shape-free and configuration coding was used. The addition of a system which directly correlated the contours of shape-free images also significantly increased recognition, suggesting extra information was still available. The coding of the configuration provided a means of automatic measurement of the face-shape using an active shape model. Taken together, these results strongly support the suggestion that faces should be considered as lying in a high-dimensional manifold which is linearly approximated by these two factors, possibly with a separate system for local features.

Abstract not found

A model describing the dynamics of the synaptic weights of a single neuron performing Hebbian learning is described. The neuron is repeatedly excited by a set of input patterns. Its response is modeled as a continuous, nonlinear function of its excitation. We study how the model forms a self-organized representation of the set of input patterns. The dynamical equations are solved directly in a few simple cases. The model is studied for random patterns by a signal to noise analysis, and by introducing a partition function and applying the replica approach. As the number of patterns is increased a first order phase transitions occurs where the neuron becomes unable to remember one pattern but rather learns to a mixture of very many patterns. The critical number of patterns for this transition scales as N b , where N is the number of synapses and b is the degree of nonlinearity. The leading order finite size corrections are calculated and compared with numerical simulations. It is shown...

NETtalk is a massively-parallel network that learns to convert English text to phonemes. In NETtalk, the memory representations are shared among many processing units, and these representations are learned by practice. In humans, distributed practice is more effective for longterm retention than massed practice, and we wondered whether learning in NETtalk had similar properties. NETtalk was tested on cued paired-associate recall using nonwords as stimuli. Retention of these target items was measured as a function of spacing, or the number of interspersed items between successive repetitions of the target. A significant advantage for spaced or distributed items was found for spacings of up to forty intervening items when tested at a retention interval of 64 items. Conversely, a significant advantage for massed items was found if testing immediately followed study. These results'are strikingly similar to the results of many experiments using human subjects and suggest an explanation based on distributed representations

In this paper we present an associative-memory-based face detection system. First, the symmetry of human faces is used to quickly locate all the candidates of human faces with all possible sizes and locations. Then two associative memories are used to decide whether or not a human face exits at the locations. Some experimental results are given. 1.

This literature review discusses the implementation of cellular automata using associative memories as the state machine. Some historical context is described, as are associative memories and brief notes on hardware implementations 1 Introduction Cellular Automata (CA) are systems of small interconnected elements, or cells. Typically they are represented as one dimensional or two dimensional grids although in theory any geometric shape is possible [66]. The cellular automata progresses through a series of states, or configurations, through successive iterations, and it is this progression that is of interest. Each of the cells in the automaton has a state which is chosen from a set of possible states. Typically the cells in an automaton are identical apart from the possibility of each having a different state. For example, in Conway's Life, a familiar example, any particular cell may either be "alive" ("on") or "dead" ("off"), there being only two states. The biological analogy is oft...

In this paper we describe five paradigms for machine learning- connectionist (neural network) methods, genetic algorithms and classifier systems, empirical methods for inducing rules and decision trees, analytic learning methods, and case-based approaches. We consider some dimensions along which these paradigms vary in their approach to learning, and then review the basic methods used within each framework, together with open research issues. We will argue that the similarities among the paradigms are more important than their differences, and that future work should at-tempt to bridge the existing boundaries. Finally, we discuss some recent developments in the field of machine learning, and speculate on their impact for both research and applications.