Abstract

Stability and encoding properties of two-layer nonlinear feedback neural networks are examined. Bidirectionality, forward and backard information flow, is introduced in neural nets to produce two-way associative search for stored associations (A, B, ). Passing information through M gives one direction; passing it through its transpose M r gives the other. A bidirectional associative memory. (BAM) behaves as a hetero- associative content addressable memory (CAM), storing and recalling the vector pairs (A1, Bi),-..,(Am Bin) , where .4 {0,1}"and B We prove that every n-by-p matrix M is a bidirectionally stable heteroas- sociative CAM for both binary/bipolar and continuous neurons a, and hi. When the BAM neurons are activated, the network quickly evolves to a stable state of two-pattern reverberation, or resonance. The stable reverberation corresponds to a system energy local minimum. Heteroassociafive inlormation is encoded iu a BAM by summing correlation matrices. The BAM storage capact .ty for reliable recall is roughly m < niin(n, p). No more heteroassociafive pairs can be 'reliably stored and recalled than the lesser of the dimensions of the pattern spaces (0,1 }"and 0,1 } P. The Appendix shos that it is better on average to use bipolar {- 1,i} coding than binary. {0,1 } coding of heteroassociative pairs (.4, B,). BAM encoding and decoding are combined in the adaptive BAM, which extends global bidirectional stabflit), to realtime unsupervised learning. Temporal patterns (AE,--., A,,) are represented as ordered lists of binary/bipolar vectors and stored in a temporal associative memory (TAM) n-by- matrix M as a limit cycle of the dynamical system. Forward recall proceeds through M, backward recall through M r . Temporal patterns are stored by summing contiguous bipolar...

Citations

2198	Fuzzy sets - Zadeh - 1965
1298	Neural networks and physical systems with emergent collective computational abilities - Hopfield - 1982
436	Neural” computation of decision optimization problems - Hopfield, Tank - 1985
359	A massively parallel architecture for a self-organizing neural pattern recognition machine - Carpenter, Grossberg - 1987
211	Absolute stability of global pattern formation and parallel memory storage by competitive neural networks - Cohen, Grossberg
100	Distinctive features, categorical perception, and probability learning: Some applications of a neural model - Anderson, Silverstein, et al. - 1977
79	Correlation matrix memories - Kohonen - 1970
78	The capacity of the Hopfield associative memory - McEliece, Posner, et al. - 1987
59	Neural dynamics of adaptive sensory-motor control: Expanded edition - Grossberg, Kuperstein - 1989
30	Cognitive and psychological computation with neural models - Anderson - 1983
26	Masking fields: A massively parallel neural architecture for learning, recognizing, and predicting multiple groupings of patterned data - Cohen, Grossberg - 1987
23	Association-A model of associative memory - Nakano - 1972
22	Some nonlinear networks capable of learning a spatial pattern of arbitrary complexity - Grossberg - 1968
17	Storage and processing of information in distributed associative memory systems - Kohonen, Oja, et al. - 1981
7	A mathematical foundation for statistical neurodynamics - Amari, Yoshida, et al. - 1977
7	Optical information processing based on associative-memory model of neural nets with thresholding and feedback - Psaltis, Farhat - 1985
5	Performance limits of optical, electro-optical, and electronics neurocomputers - Hecht-Nielsen - 1986
2	A Template Matching Model for Pattern Recognition: Self-Organization of Templates and Template Matching by a Disinhlbltory Neural Network - Hirai - 1980
1	Categorization and selective neurons, " in Parallel Models of Associative - Anderson, Mozer - 1981
1	the serial leanting of lists - On - 1969
1	does a bran build a cognitive code9 - How - 1980
1	A logical calculus of the ideas immanent in netwous activity - McCulloch, Pitts - 1943