## Sparse Distributed Memory and Related Models (1993)

### Cached

### Download Links

Venue: | Associative Neural Memories |

Citations: | 43 - 2 self |

### BibTeX

@INPROCEEDINGS{Kanerva93sparsedistributed,

author = {Pentti Kanerva},

title = {Sparse Distributed Memory and Related Models},

booktitle = {Associative Neural Memories},

year = {1993},

pages = {50--76},

publisher = {Oxford University Press}

}

### Years of Citing Articles

### OpenURL

### Abstract

This chapter describes one basic model of associative memory, called the sparse distributed memory, and relates it to other models and circuits: to ordinary computer memory, to correlation-matrix memories, to feed-forward artificial neural nets, to neural circuits in the brain, and to associative-memory models of the cerebellum. Presenting the various designs within one framework will hopefully help the reader see the similarities and the differences in designs that are often described in different ways.

### Citations

1579 | Neural networks and physical systems with emergent collective computational abilities - Hopfield - 1982 |

1306 |
Self-Organization and Associate Memory
- Kohonen
- 1984
(Show Context)
Citation Context ...le copies of target words in memory, and of retrieving the most likely target word based on the majority rule, are found already in the redundant hash addressing method of Kohonen and Reuhkala (1978, =-=Kohonen 1980-=-). The method of realizing these ideas in redundant hash addressing is very different from their realization in a sparse distributed memory. Retrieval and memory capacity will be analyzed statisticall... |

288 | Self-Organization and Associative - Kohonen - 1984 |

286 | Principles of Neurodynamics - Rosenblatt - 1962 |

245 | A Theory of Cerebellar Cortex - Marr - 1969 |

221 |
Sparse distributed memory
- Kanerva
- 1988
(Show Context)
Citation Context ...ften described in different ways. 3.1.1.Sparse Distributed Memory as a Model of Human Long-Term Memory Sparse Distributed Memory (SDM) was developed as a mathematical model of human long-term memory (=-=Kanerva 1988-=-). The pursuit of a simple idea led to the discovery of the model, namely, that the distances between concepts in our minds correspond to the distances between points of a high-dimensional space. In w... |

219 | Outline for a theory of intelligence - Albus - 1991 |

180 | Theory of Cerebellar Function - Albus |

141 |
NETtalk: A parallel network that learns to read aloud
- Sejnowski
- 1986
(Show Context)
Citation Context ...agreement with that, Joglekar (1989) experimented with NETtalk data and got his best results by using a subset of the data addresses as hard addresses (NETtalk transcribes English text into phonemes; =-=Sejnowski and Rosenberg 1986-=-). In a series of experiments by Danforth (1990), recognition of spoken digits, encoded in 240 bits, improved dramatically when uniformly random hard addresses were replaced by addresses that represen... |

88 |
Correlation Matrix Memories
- Kohonen
- 1972
(Show Context)
Citation Context ...memory. The Y-variables are derived from the X-variables (each Y-variable compares the data addresses X to a specific hard address), whereas in the original correlationmatrix memories (Anderson 1968; =-=Kohonen 1972-=-), the X-variables are used directly, and the variables are continuous. Changing from the X-variables to the Yvariables means, mathematically, that the input dimension is blown way up (from a thousand... |

19 | A class of designs for a sparse distributed memory - Jaeckel - 1989 |

19 | Self Organization and Associative Memory, Second Edition - Kohonen - 1984 |

15 | The modified Kanerva model for automatic speech recognition - Prager, Fallside - 1989 |

13 |
The cerebellum and neuronal control
- Ito
- 1984
(Show Context)
Citation Context ...f neurons of only six major kinds. Its morphology has been studied extensively since early 1900s, its role in fine motor control has been established, and its physiology is still studied intensively (=-=Ito 1984-=-). The cortex of the cerebellum is sketched in Figure 2-11 after Llin_is (1975). Figure 2-12 is Figure 2-9 redrawn in an orientation that corresponds to the sketch of the cerebellar cortex. (( FIGURE ... |

11 | Comparison between Kanerva’s SDM and Hopfield-type neural networks - Keeler - 1988 |

10 |
Predicting weather using a genetic memory: A combination of Kanerva’s sparse distributed memory and Holland’s genetic algorithms
- Rogers
- 1990
(Show Context)
Citation Context ...ng Net for working with nonstationary data. The use of continuous variables by Prager and Fallside has been discussed in Section 3.6.4. Sparse distributed memory has been simulated on many computers (=-=Rogers 1990-=-b), including the highly parallel Connection Machine (Rogers 1989b) and special-purpose neural-network computers (Nordstrm 1991). Hardware implementations have used standard logic circuits and memory ... |

6 | Learning to read aloud: A neural network approach using sparse distributed memory - Joglekar - 1989 |

6 |
Kanerva's sparse distributed memory: An associative memory algorithm well-suited to the Connection Machine
- Rogers
- 1989
(Show Context)
Citation Context ... variables by Prager and Fallside has been discussed in Section 3.6.4. Sparse distributed memory has been simulated on many computers (Rogers 1990b), including the highly parallel Connection Machine (=-=Rogers 1989-=-b) and special-purpose neural-network computers (Nordstrm 1991). Hardware implementations have used standard logic circuits and memory chips (Flynn et al. 1987) and programmable gate arrays (Saarinen ... |

5 | The capacity of the Kanerva associative memory - Chou - 1989 |

5 |
Assigning meaning to data: using sparse distributed memory for multilevel cognitive tasks
- Manevitz, Zemach
- 1997
(Show Context)
Citation Context ...nformation about a variable in weather data. Other research issues include the storage of sequences (Manevitz 1991) and the KANERVA/SDM AND RELATED MODELS/02/02/02/P. 34 hierarchical storage of data (=-=Manevitz and Zemach 1997-=-). Most studies of sparse distributed memory have used binary data and have dealt with multivalued variables by encoding them according to an appropriate binary code. Table 3.1 is an example of such a... |

4 | An empirical investigation of sparse distributed memory using discrete speech recognition - Danforth - 1990 |

4 | A very fast associative method for the recognition and correction of misspelt words, based on redundant hash addressing - Kohonen, Reuhkala - 1978 |

4 |
Total recall in distributed associative memories
- Danforth
- 1991
(Show Context)
Citation Context ... may be mostly on, some may be mostly off, and some may depend on others. It is possible to analyze the data (X, Z) and the hard addresses A and to determine optimal storage and retrieval algorithms (=-=Danforth 1991-=-), but we can also use iterative training by error correction, as described above for Albus' CMAC. This was done by Joglekar and by Danforth in their above-mentioned experiments. When error correction... |

3 |
High performance recording algorithm for Hopfield model associative memories. Optical Engineering 28(1):46--54
- Hassoun, Youssef
- 1989
(Show Context)
Citation Context ...6--48). High probability of activation (ps0.5) works poorly with the outer-product leaning rule. However, it is appropriate for an analytic solution to storage by the Ho--Kashyap recording algorithm (=-=Hassoun and Youssef 1989-=-). This algorithm finds a contents matrix C that solves the linear inequalities implied by Z = W,where W is the matrix of data words to be stored, and Z = z (S) = z(YC) is the matrix of words retrieve... |

3 |
Statistical prediction with Kanerva’s sparse distributed memory
- Rogers
- 1988
(Show Context)
Citation Context ... variables by Prager and Fallside has been discussed in Section 3.6.4. Sparse distributed memory has been simulated on many computers (Rogers 1990b), including the highly parallel Connection Machine (=-=Rogers 1989-=-b) and special-purpose neural-network computers (Nordstrm 1991). Hardware implementations have used standard logic circuits and memory chips (Flynn et al. 1987) and programmable gate arrays (Saarinen ... |

2 | The Marr and Albus theories of the cerebellum: Two early models of associative memory - Albus - 1989 |

2 | The modified Kanerva model: Theory and results for real-time word recognition - Clarke, Prager, et al. - 1991 |

2 | Effective packing of patterns in sparse distributed memory by selective weighting of input bits - Kanerva - 1991 |

2 | Kanerva's Sparse Distributed Memory with Multiple Hamming Thresholds - Pohja, Kaski - 1992 |

2 |
Designing and Using Massively Parallel Computers for Artificial Neural Networks
- Nordström, Svensson
- 1992
(Show Context)
Citation Context ...ion 3.6.4. Sparse distributed memory has been simulated on many computers (Rogers 1990b), including the highly parallel Connection Machine (Rogers 1989b) and special-purpose neural-network computers (=-=Nordström 1991-=-). Hardware implementations have used standard logic circuits and memory chips (Flynn et al. 1987) and programmable gate arrays (Saarinen et al. 1991a). A systolic-array implementation of sparse distr... |

1 |
Method and Apparatus for Implementation of the CMAC Mapping Algorithm
- Albus
- 1980
(Show Context)
Citation Context ...the edge of the space). The systematic placement of the hard locations allows addresses to be converted into activation vectors very efficiently in a hardware realization or in a computer simulation (=-=Albus 1980-=-). Correspondence of the hard locations to the granule cells of the cerebellum is RK/ N 1 R 1 -- ()K/ + () N RK/ N KANERVA/SDM AND RELATED MODELS/02/02/02/P. 31 natural in Albus' model. To make the mo... |

1 |
A memory storage module utilizing spatial correlation functions. Kybernetik 5(3):113--119
- Anderson
- 1968
(Show Context)
Citation Context ...elation-matrix memory. The Y-variables are derived from the X-variables (each Y-variable compares the data addresses X to a specific hard address), whereas in the original correlationmatrix memories (=-=Anderson 1968-=-; Kohonen 1972), the X-variables are used directly, and the variables are continuous. Changing from the X-variables to the Yvariables means, mathematically, that the input dimension is blown way up (f... |

1 | Complete report, with the same title, in RIACS - Norwell - 1991 |

1 |
Sparse Distributed Memory Prototype: Principles of Operation
- Flynn, Kanerva, et al.
- 1987
(Show Context)
Citation Context ...luding the highly parallel Connection Machine (Rogers 1989b) and special-purpose neural-network computers (Nordstrm 1991). Hardware implementations have used standard logic circuits and memory chips (=-=Flynn et al. 1987-=-) and programmable gate arrays (Saarinen et al. 1991a). A systolic-array implementation of sparse distributed memory and a resistor circuit for computing the Hamming distances have been described by K... |

1 | Two-level neural network for deterministic logic processing - Hassoun - 1988 |

1 | Two Alternate Proofs of Wang's Lune Formula for Sparse Distributed Memory and an Integral Approximation - KANERVASDM, MODELS020202P - 1988 |

1 | Contour-map encoding of shape for early vision - Kanerva - 1990 |

1 | Notes on Implementation of Sparse Distributed Memory - Keeler, Denning - 1986 |

1 | The cortex of the cerebellum. Scientific American 232(1):56--71 - Llins - 1975 |

1 | Intelligent network management and functional cerebellum synthesis - Loebner - 1989 |

1 |
Designing and Using Massively Parallel Computers for Artificial Neural Networks. Licentiate thesis 1991:12L
- Nordstrm
- 1991
(Show Context)
Citation Context ...ion 3.6.4. Sparse distributed memory has been simulated on many computers (Rogers 1990b), including the highly parallel Connection Machine (Rogers 1989b) and special-purpose neural-network computers (=-=Nordstrm 1991-=-). Hardware implementations have used standard logic circuits and memory chips (Flynn et al. 1987) and programmable gate arrays (Saarinen et al. 1991a). A systolic-array implementation of sparse distr... |

1 |
BIRD: A General Interface for Sparse Distributed memory Simulators
- Rogers
- 1990
(Show Context)
Citation Context ...ng Net for working with nonstationary data. The use of continuous variables by Prager and Fallside has been discussed in Section 3.6.4. Sparse distributed memory has been simulated on many computers (=-=Rogers 1990-=-b), including the highly parallel Connection Machine (Rogers 1989b) and special-purpose neural-network computers (Nordstrm 1991). Hardware implementations have used standard logic circuits and memory ... |

1 |
Highly parallel hardware implementation of sparse distributed memory
- Saarinen, Lindell, et al.
- 1991
(Show Context)
Citation Context ...gers 1989b) and special-purpose neural-network computers (Nordstrm 1991). Hardware implementations have used standard logic circuits and memory chips (Flynn et al. 1987) and programmable gate arrays (=-=Saarinen et al. 1991-=-a). A systolic-array implementation of sparse distributed memory and a resistor circuit for computing the Hamming distances have been described by Keeler and Denning (1986). 3.9.Associative Memory as ... |

1 |
Self-organization with Kanerva's sparse distributed memory
- Saarinen, Pohja, et al.
- 1991
(Show Context)
Citation Context ...gers 1989b) and special-purpose neural-network computers (Nordstrm 1991). Hardware implementations have used standard logic circuits and memory chips (Flynn et al. 1987) and programmable gate arrays (=-=Saarinen et al. 1991-=-a). A systolic-array implementation of sparse distributed memory and a resistor circuit for computing the Hamming distances have been described by Keeler and Denning (1986). 3.9.Associative Memory as ... |

1 | Two Alternate Proofs of Wang’s Lune Formula for Sparse Distributed Memory and an Integral Approximation - Jaeckel - 1988 |

1 | The cortex of the cerebellum. Scientific American 232(1):56–71 - Llinás - 1975 |

1 | P.1990.Contour-map encoding ofshapeforearlyvision.InD.S.Touretzky Information Processing Systems 2:282-289 (Proceedings NIPS-89 - Kanerva - 1991 |

1 | The cortex of the cerebellum. Scientific American 232(1):56--71 - Llinfis - 1975 |

1 | Implementing a "sense of time" via entropy in associative memories - Manevitz - 1991 |

1 | Designing and Using Massively Parallel Computers for Artificial Neural Networks - Nordslr6m - 1991 |