Abstract

In order to solve the "sensitivity-stability" problem --- and its immediate correlate, the problem of sequential learning --- it is crucial to develop connectionist architectures that are simultaneously sensitive to, but not excessively disrupted by, new input. French (1992) suggested that to alleviate a particularly severe form of this disruption, catastrophic forgetting, it was necessary for networks to dynamically separate their internal representations during learning. McClelland, McNaughton, & O'Reilly (1995) went even further. They suggested that nature's way of implementing this obligatory separation was the evolution of two separate areas of the brain, the hippocampus and the neocortex. In keeping with this idea of radical separation, a "pseudo-recurrent" memory model is presented here that partitions a connectionist network into two functionally distinct, but continually interacting areas. One area serves as a final-storage area for representations; the other is an e...

Citations

1313	Finding structure in time - Elman - 1990
721	Organization of Behavior - Hebb - 1949
662	Repository of machine learning databases - Murphy, Aha - 1994
288	Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and - McClelland, McNaughton, et al. - 1995
210	Neural Darwinism: The theory of neuronal group selection - Edelman - 1987
192	ART2: self-organization of stable category recognition codes for analog input patterns - Carpenter, Grossberg - 1987
170	Sparse Distributed Memory - Kanerva - 1988
146	Catastrophic interference in connectionist networks: The sequential learning problem. In The psychology of learning and motivation - McCloskey, NJ - 1989
76	Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions - Ratcliff - 1990
75	A retrieval model for both recognition and recall - Gillund, Schiffrin - 1984
48	Human category learning: Implications for backpropagation models - Kruschke - 1993
31	Semi-distributed representations and catastrophic forgetting in connectionist networks - French - 1992
31	Catastrophic Forgetting, rehearsal, and pseudorehearsal - Robins - 1995
30	A distributed memory model for serial-order information - Murdock - 1983
24	Retrieval processes in recognition memory - Ratcliff, Murdock - 1976
24	The list-strength effect: I. Data and discussion - Ratcliff, Clark, et al. - 1990
23	Episodic memory in connectionist networks - Kortge - 1990
20	Catastrophic interference is eliminated in pretrained networks - McRae, Hetherington - 1993
18	Gradual unlearning and catastrophic interference: A comparison of distributed architectures - Lewandowsky - 1991
17	Dynamically constraining connectionist networks to produce distributed, orthogonal representations to reduce catastrophic interference - French - 1994
17	An analysis of catastrophic interference - Sharkey, Sharkey - 1995
16	List-strength effect: II. Theoretical mechanisms - Shiffrin, Ratcliff, et al. - 1990
15	Factors influencing speed and accuracy of word recognition - Atkinson - 1973
11	The effects of pattern presentation on interference in backpropagation networks - Murre - 1992
9	Comparison between Kanerva's SDM and Hopfield-type neural networks - Keeler - 1988
8	Consolidation and Learning: A Connectionist Model of Human Credit Assignment - Chown - 1993
8	Development of exclusivity in perceptually based categories of young infants - Eimas, Quinn - 1994
7	The sequential learning problem in connectionist networks - Hetherington - 1991
7	A connectionist account of interference effects in early infant memory and categorization - Mareschal, French - 1997
6	A composite holographic associative recall model - Metcalfe-Eich - 1982
2	Catastrophic Forgetting in Neural Networks: A Review and an Analysis of the Pseudorehearsal Solution. Otago University CS tech report AIM-36-97-2 - Frean, Robins - 1996
1	Is there “catastrophic interference” in connectionist networks - Psychology, University, et al. - 1989
1	Interference and the Representation of Events - Murnane, Shiffrin - 1991