How do people know as much as they do with as little information as they get? The problem takes many forms; learning vocabulary from text is an especially dramatic and convenient case for research. A new general theory of acquired similarity and knowledge representation, latent semantic analysis (LSA), is presented and used to successfully simulate such learning and several other psycholinguistic phenomena. By inducing global knowledge indirectly from local co-occurrence data in a large body of representative text, LSA acquired knowledge about the full vocabulary of English at a comparable rate to schoolchildren. LSA uses no prior linguistic or perceptual similarity knowledge; it is based solely on a general mathematical learning method that achieves powerful inductive effects by extracting the right number of dimensions (e.g., 300) to represent objects and contexts. Relations to other theories, phenomena, and problems are sketched. Prologue "How much do we know at any time? Much more, or so I believe, than we know we know!" —Agatha Christie, The Moving Finger A typical American seventh grader knows the meaning of

This article presents a theory in which automatization is construed as the acquisition of a domain-specific knowledge base, formed of separate representations, instances, of each exposure to the task. Processing is considered automatic if it relies on retrieval of stored instances, which will occur only after practice in a consistent environment. Practice is important because it increases the amount retrieved and the speed of retrieval; consistency is important because it ensures that the retrieved instances will be useful. The theory accounts quantitatively for the power-function speed-up and predicts a power-function reduction in the standard deviation that is constrained to have the same exponent as the power function for the speed-up. The theory accounts for qualitative properties as well, explaining how some may disappear and others appear with practice. More generally, it provides an alternative to the modal view of automaticity, arguing that novice performance is limited by a lack of knowledge rather than a scarcity of resources. The focus on learning avoids many problems with the modal view that stem from its focus on resource limitations. Automaticity is an important phenomenon in everyday men-tal life. Most of us recognize that we perform routine activities quickly and effortlessly, with little thought and conscious aware-ness--in short, automatically (James, 1890). As a result, we of-ten perform those activities on "automatic pilot " and turn our minds to other things. For example, we can drive to dinner while conversing in depth with a visiting scholar, or we can make coffee while planning dessert. However, these benefits may be offset by costs. The automatic pilot can lead us astray, caus-ing errors and sometimes catastrophes (Reason & Myceilska, 1982). If the conversation is deep enough, we may find ourselves and the scholar arriving at the office rather than the restaurant, or we may discover that we aren't sure whether we put two or three scoops of coffee into the pot. Automaticity is also an important phenomenon in skill acqui-sition (e.g., Bryan & Harter, 1899). Skills are thought to consist largely of collections of automatic processes and procedures

Associative memories are conventionally used to represent data with very simple structure: sets of pairs of vectors. This paper describes a method for representing more complex compositional structure in distributed representations. The method uses circular convolution to associate items, which are represented by vectors. Arbitrary variable bindings, short sequences of various lengths, simple framelike structures, and reduced representations can be represented in a fixed width vector. These representations are items in their own right, and can be used in constructing compositional structures. The noisy reconstructions extracted from convolution memories can be cleaned up by using a separate associative memory that has good reconstructive properties.

Distributed representations are attractive for a number of reasons. They offer the possibility of representing concepts in a continuous space, they degrade gracefully with noise, and they can be processed in a parallel network of simple processing elements. However, the problem of representing nested structure in distributed representations has been for some time a prominent concern of both proponents and critics of connectionism [Fodor and Pylyshyn 1988; Smolensky 1990; Hinton 1990]. The lack of connectionist representations for complex structure has held back progress in tackling higher-level cognitive tasks such as language understanding and reasoning. In this thesis I review connectionist representations and propose a method for the distributed representation of nested structure, which I call "Holographic Reduced Representations " (HRRs). HRRs provide an implementation of Hinton's [1990] "reduced descriptions". HRRs use circular convolution to associate atomic items, which are rep...

structure. Perceptual classification learning experiments were conducted in which presentation frequencies of individual exemplars were manipulated. The exemplars had varying degrees of similarity to members of the target and contrast categories. Classification accuracy and typicality ratings increased for exemplars presented with high frequency and for members of the target category that were similar to the high-frequency exemplars. Typicality decreased for members of the contrast category that were similar to the high-frequency exemplars. A frequency-sensitive similarity-to-exemplars model provided a good quantitative account of the classification learning and typicality data. The interactive relations among similarity, frequency, and categorization are considered in the General Discussion. Among the most well-established findings in the categorization literature is that categories have "graded structures"

A solution to the problem of representing compositional structure using distributed representations is described. The method uses circular convolution to associate items, which are represented by vectors. Arbitrary variable bindings, short sequences of various lengths, frames, and reduced representations can be compressed into a fixed width vector. These representations are items in their own right, and can be used in constructing compositional structures. The noisy reconstructions given by convolution memories can be cleaned up by using a separate associative memory that has good reconstructive properties.

Free recall and recognition are simulated in a network model of the hippocampal formation, incorporating simplified simulations of neurons, synaptic connections, and the effects of acetylcholine. Simulations focus on modeling the effects of the acetylcholine receptor blocker scopolamine on human memory. Systemic administration of scopolamine is modeled by blockade of the cellular effects of acetylcholine in the model, resulting in memory impairments replicating data from studies on human subjects. This blockade of cholinergic effects impairs the encoding of new input patterns (as measured by delayed free recall), but does not impair the delayed free recall of input patterns learned before the blockade. The impairment is selective to the free recall but not the recognition of items encoded under the influence of scopolamine. In the model, scopolamine blocks strengthening of recurrent connections in region CA3 to form attractor states for new items (encoding impaired) but allows recurrent excitation to drive the network into previously stored attractor states (retrieval spared). Neuron populations representing items (individual words) have weaker recurrent connections than neuron populations representing experimental context. When scopolamine further weakens the strength of recurrent connections it selectively prevents the subsequent reactivation of item attractor states by context input (impaired free recall) without impairing the subsequent reactivation of context attractor states by item input (spared recognition). This asymmetry in the strength of attractor states also allows simulation of the list-strength effect for free recall but not recognition. Simulation of a paired associate learning paradigm predicts that scopolamine should greatly enhance proactive interfere...

Abstract not found

: Holographic Reduced Representations (HRRs) are a method for encoding nested relational structures in fixed width vector representations. HRRs encode relational structures as vector representations in such a way that the superficial similarity of the vectors reflects both superficial and structural similarity of the relational structures. HRRs also support a number of operations that could be very useful in psychological models of human analogy processing: fast estimation of superficial and structural similarity via a vector dot-product; finding corresponding objects in two structures; and chunking of vector representations. Although similarity assessment and discovery of corresponding objects both theoretically take exponential time to perform fully and accurately, with HRRs one can obtain approximate solutions in constant time. The accuracy of these operations with HRRs mirrors patterns of human performance on analog retrieval and processing tasks. Keywords: neural networks, distributed representations, binding, analogy, analog retrieval, structure, chunking, systematicity 1

This article presents an attractor neural network model of paired-associate learning and uses a model-based analysis of experimental data to Neural Computation 13, 2075--2092 (2001) c 2001 Massachusetts Institute of Technology shed light on some basic unresolved questions concerning the nature of associations in human memory. The paired-associate learning task is one of the standard assays of human episodic memory. Typically subjects are presented with randomly paired items (e.g., words, letter strings, pictures) and asked to remember each A-B pair for a subsequent memory test. At test, the A items are presented as cues, and subjects attempt to recall the appropriate B items