A parallel distributed processing model of visual word recognition and pronunciation is described. The model consists of sets of orthographic and phonologlc ~ units and an interlevel of hidden units. Weights on connections between units were modified during a training phase using the back-propa-gation learning algorithm. The model simulates many aspects of human performance, including (a) differences bet~n~.'n words in terms of processing difficulty, (b) pronunciation of novel items, (c) differences between readers in terms of word recognition skill, (d) transitions from beginning to skilled reading, and (e) differences in performance on lexieal decision and naming tasks. The model's behavior early in the learning phase corresponds to that of children acquiring word recognition skills. Training with a smaller number of hidden units produces output characteristic of many dys-lexic readers. Naming is simulated without pronunciation rules, and lexical decisions are simulated without accessing word-level representations. The performance of the model is largely determined by three factors: the nature of the input, a significant fragment of written English; the learning rule, which encodes the implicit structure of the orthography in the weights on connections; and the architecture of the system, which influences the scope of what can be learned. The recognition and pronunciation of words is one of the cen-

Abstract. In this paper three problems for a connectionist account of language are considered: 1. What is the nature of linguistic representations? 2. How can complex structural relationships such as constituent structure be represented? 3. How can the apparently open-ended nature of language be accommodated by a fixed-resource system? Using a prediction task, a simple recurrent network (SRN) is trained on multiclausal sentences which contain multiply-embedded relative clauses. Principal component analysis of the hidden unit activation patterns reveals that the network solves the task by developing complex distributed representations which encode the relevant grammatical relations and hierarchical constituent structure. Differences between the SRN state representations and the more traditional pushdown store are discussed in the final section.

threw " could either refer to toss or host; and "ball " could refer to a sphere or a dance, How are the appropriate meanings selected so that a single, coherent interpretation of the sentence is produced? Vague words also present difficulties. In the sentences The container held the apples " and "The container held the cola, " the word "container " refers to two different objects (1). How does the context affect the interpretation of vague words? A third problem is the complexity of assigning the correct thematic roles (9) to the objects referred to in a sentence. Consider:

this article, in conjunction with those developed previously (Plaut et al., 1996; Seidenberg & McClelland, 1989), illustrate how connectionist computational principles---distributed representation, structure-sensitive learning, and interactivity---can provide insight into central empirical phenomena in normal and impaired lexical processing. Moreover, they make it clear that distinctions in the function of the lexical system---as manifest in the behaviour of experimental subjects--- need not re#ect corresponding distinctions in the structure of the system. Thus, networks exhibit word-frequency effects and word/nonword discrimination without word representations, and spelling --sound consistency effects without separate mechanisms for regular and exception items. In this way, gaining insight into the structure and function of the cognitive system by observing its normal and impaired behaviour ---the central goal of cognitive psychology and neuropsycho logy---may depend critically on developing theories and explicit simulations in the context of a speci#c computational framework that relates structure to function

Representations for semantic information about words are necessary for many applications of neural networks in natural language processing. This paper describes an efficient, corpus-based method for inducing distributed semantic representations for a large number of words (50,000) from lexical coccurrence statistics by means of a large-scale linear regression. The representations are successfully applied to word sense disambiguation using a nearest neighbor method.

A distributed attractor network is trained on an abstract version of the task of deriving the meanings of written words. When processing a word, the network starts from the final activity pattern of the previous word. Two words are semantically related if they overlap in their semantic features, whereas they are associatively related if one word follows the other frequently during training. After training, the network exhibits two empirical effects that have posed problems for distributed network theories: much stronger associative priming than semantic priming, and significant associative priming across an intervening unrelated item. It also reproduces the empirical findings of greater priming for low-frequency targets, degraded targets, and high-dominance category exemplars.

the properties of distributed network models, and support this account by demonstrating that an implemented simulation closely approximates the empirical findings despite the absence of expectancy-based processes and postlexical semantic matching. The results suggest that distributed network models can provide a viable single-mechanism account of lexical processing. Introduction It is well-established that people are faster and more accurate to read a word (e.g., BUTTER) when it is preceded by a related word (e.g., BREAD) compared with when it is preceded by an unrelated word (e.g., DOCTOR; The research was supported by an NIMH FIRST award (MH55628) to the first author and by NIMH Training Grant 5T32MH19102 and NICHD Grant 80258. The computational simulation was run using customized software written within the Xerion simulator (version 3.1) developed by Drew van Camp, Tony Plate, and Geoff Hinton at the Univers

Measures of the speed andacg&U:W ofproc:)gH3fiPW&gcc with nonadjacgH dependencH3 derived from the response -signal speed-acgnalg tradeo#proco#g& were used to examine the nature of the memory system that underlies sentenc ctencesg):WP Three experiments with di#erent sentenc strucg9P demonstrated that theacg))W& of procWPPgH3 dependenc decndenc as more material was interpolated betweennonadjac33 cnadjac33(g However, procer,g):WPWg wasuna#ecUP by the amount of interpolated material,indicl,g39UPfi memory representations for previously procious ciouslygfi9 cfi beac9((fi9 direcfi9g These results suggest that acfi99WgH339fi)gcgc memory system mediates sentenc ctenc&(3:WgH inwhic syntac9g andsemantic information providedirec acec to memory representations without the need tosearc through extraneous representations. Notably,cably,g3Pfi3)gH3&(:&gc appears to underlie the interpretation ofsentenc strucgH9 that also require therec33&U of order information, a type of operation that has been shown tonec:&(gH9fi a slowsearc proc3 in list-learningexperiments(Mct-lea 2001;Mc1;g) & Dosher, 1993).

DISLEX is an artificial neural network model of the mental lexicon. It was built to test computationally whether the lexicon could consist of separate feature maps for the different lexical modalities and the lexical semantics, connected with ordered pathways. In the model, the orthographic, phonological, and semantic feature maps and the associations between them are formed in an unsupervised process, based on cooccurrence of the lexical symbol and its meaning. After the model is organized, various damage to the lexical system can be simulated, resulting in dyslexic and category-specific aphasic impairments similar to those observed in human patients. 1 Introduction The human lexical system is believed to be highly modular, consisting of a central semantic component and separate symbol memories for the different input and output modalities (Caramazza 1988; McCarthy and Warrington 1990). Such an architecture is intuitively compelling since the modalities give rise to different repres...

Ambiguous words are processed more quickly than unambiguous words in a lexical decision task despite the fact that each sense of an ambiguous word is less frequent than the single sense of unambiguous words of equal frequency or familiarity. In this computer simulation study, we examined the effects of different assumptions of a fully recurrent connectionist model in accounting for this processing advantage for ambiguous words. We argue that the ambiguity advantage effect can be accounted for by distributed models if (a) the least mean square (LMS) error-correction algorithm rather than the Hebbian algorithm is used in training the network and (b) activation of the units representing the spelling rather than the meaning is used to index word recognition times. An important advantage of computational models is that the underlying assumptions of the model must be explicitly formulated. This explicit formulation allows comparison of assumptions that are highly similar. In some cases, virtually identical assumptions can give rise to qualitative differences rather than merely quantitative differences. In this article,