Stability and encoding properties of two-layer nonlinear feedback neural networks are examined. Bidirectionality, forward and backard information flow, is introduced in neural nets to produce two-way associative search for stored associations (A, B, ). Passing information through M gives one direction; passing it through its transpose M r gives the other. A bidirectional associative memory. (BAM) behaves as a hetero- associative content addressable memory (CAM), storing and recalling the vector pairs (A1, Bi),-..,(Am Bin) , where .4 {0,1}"and B We prove that every n-by-p matrix M is a bidirectionally stable heteroas- sociative CAM for both binary/bipolar and continuous neurons a, and hi. When the BAM neurons are activated, the network quickly evolves to a stable state of two-pattern reverberation, or resonance. The stable reverberation corresponds to a system energy local minimum. Heteroassociafive inlormation is encoded iu a BAM by summing correlation matrices. The BAM storage capact .ty for reliable recall is roughly m < niin(n, p). No more heteroassociafive pairs can be 'reliably stored and recalled than the lesser of the dimensions of the pattern spaces (0,1 }"and 0,1 } P. The Appendix shos that it is better on average to use bipolar {- 1,i} coding than binary. {0,1 } coding of heteroassociative pairs (.4, B,). BAM encoding and decoding are combined in the adaptive BAM, which extends global bidirectional stabflit), to realtime unsupervised learning. Temporal patterns (AE,--., A,,) are represented as ordered lists of binary/bipolar vectors and stored in a temporal associative memory (TAM) n-by- matrix M as a limit cycle of the dynamical system. Forward recall proceeds through M, backward recall through M r . Temporal patterns are stored by summing contiguous bipolar...

xi 1 Introduction 1 1.1 Problem Statement : : : : : : : : : : : : : : : : : : : : : : : : : 6 1.2 The Neuron : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7 1.2.1 Biological Model : : : : : : : : : : : : : : : : : : : : : : 7 1.2.2 Computational Model : : : : : : : : : : : : : : : : : : : 9 1.3 Implementation Technologies : : : : : : : : : : : : : : : : : : : : 11 1.4 State of the Art : : : : : : : : : : : : : : : : : : : : : : : : : : : 14 1.5 Summary of Results : : : : : : : : : : : : : : : : : : : : : : : : 16 2 Implementation of Neural Models with Static Links 19 2.1 ANN Models with Static Links : : : : : : : : : : : : : : : : : : 20 2.1.1 The Hopfield Model : : : : : : : : : : : : : : : : : : : : : 21 2.1.2 The Perceptron Model : : : : : : : : : : : : : : : : : : : 23 2.1.3 The Multi-Layer Model : : : : : : : : : : : : : : : : : : : 24 2.2 Basic Computational Requirements : : : : : : : : : : : : : : : : 25 2.2.1 Search Phase Computations : : : : : : : : : : : : : : ...

This paper investigates the architectural requirements in simulating large neural networks using a highly parallel multiprocessor with distributed memory and optical interconnects. First, we model the structure of a neural network and the functional behavior of individual cells. These models are used to estimate the volume of messages that need to be exchanged among physical processors to simulate the weighted connections of the neural network. The distributed processor/memory organization is tailored to an electronic implementation for greater versatility and flexibility. Optical interconnects are used to satisfy the interprocessor communication bandwidth demands. The hybrid implementation attempts to balance the processing, memory and bandwidth demands in simulating asynchronous, value-passing models for cooperative parallel computation with self-learning capabilities. 1. INTRODUCTION Connectionist models of computation 1 have been in the limelight as promising alternatives to tr...

ed in Case-Based Reasoning: Papers from the 1993 Workshop, July 11-12, Washington, D.C., Technical Report WS-93-01, AAAI Press THE UTILITY PROBLEM IN CASE BASED REASONING ANTHONY G. FRANCIS ASHWIN RAM College of Computing College of Computing Georgia Institute of Technology Georgia Institute of Technology Atlanta, Georgia 30332-0280 Atlanta, Georgia 30332-0280 (404) 351-4574 (404) 853-9372 centaur@cc.gatech.edu ashwin@cc.gatech.edu ABSTRACT Case-based reasoning systems may suffer from the utility problem, which occurs when knowledge learned in an attempt to improve a system's performance degrades performance instead. One of the primary causes of the utility problem is the slowdown of conventional memories as the number of stored items increases. Unrestricted learning algorithms can swamp their memory system, causing the system to slow down more than the average speedup provided by individual learned rules. Massive parallelism is often offered as a solution to this problem. However,...

This paper investigates the architectural requirements for simulating neural networks using massively parallel multiprocessors. First, we model the connectivity patterns in large neural networks. A distributed processor/memory organization is developed for efficiently simulating asynchronous, value-passing connectionist models. Based on the network connectivity and mapping policy, we estimate the volume of messages that need to be exchanged among physical processors for simulating the weighted connections of a neural network. This helps determine the interprocessor communication bandwidth required, and the optimal number and granularity of processors needed to meet a particular cost/performance goal. The suitability of existing computers is assessed in the light of estimated architectural demands. The structural model offers an efficient methodology for mapping virtual neural networks onto a real parallel computer. It makes possible the execution of large-scale neural networks on a mod...