The problems of access -- retrieving linguistic structure from some mental grammar -- and disambiguation -- choosing among these structures to correctly parse ambiguous linguistic input -- are fundamental to language understanding. The literature abounds with psychological results on lexical access, the access of idioms, syntactic rule access, parsing preferences, syntactic disambiguation, and the processing of garden-path sentences. Unfortunately, it has been difficult to combine models which account for these results to build a general, uniform model of access and disambiguation at the lexical, idiomatic, and syntactic levels. For example psycholinguistic theories of lexical access and idiom access and parsing theories of syntactic rule access have almost no commonality in methodology or coverage of psycholinguistic data. This paper presents a single probabilistic algorithm which models both the access and disambiguation of linguistic knowledge. The algorithm is based on a parallel parser which ranks constructions for access, and interpretations for disambiguation, by their conditional probability. Low-ranked constructions and interpretations are pruned through beam-search; this pruning accounts, among other things, for the garden-path effect. I show that this motivated probabilistic treatment accounts for a wide variety of psycholinguistic results, arguing for a more uniform representation of linguistic knowledge and for the use of probabilisticallyenriched grammars and interpreters as models of human knowledge of and processing of language.

Abstract not found

Introduction Despite considerable diversity among theories about how humans process language, there are a number of fundamental assumptions which are shared by most such theories. This consensus extends to the very basic question about what counts as a cognitive process. So although many cognitive scientists are fond of referring to the brain as a `mental organ' (e.g., Chomsky, 1975)---implying a similarity to other organs such as the liver or kidneys---it is also assumed that the brain is an organ with special properties which set it apart. Brains `carry out computation' (it is argued)

This thesis presents NL-Soar, a detailed computational model of human sentence comprehension that accounts for a broad range of psycholinguistic phenomena. NL-Soar provides in-depth accounts of structural ambiguity resolution, garden path effects, unproblematic ambiguities, parsing breakdown on difficult embeddings, acceptable embeddings, immediacy of interpretation, and the time course of comprehension. The model explains a variety of both modular and interactive effects, and shows how learning can affect ambiguity resolution behavior. In addition to accounting for the qualitative phenomena surrounding parsing breakdown and garden path effects, NL-Soar explains a wide range of contrasts between garden paths and unproblematic ambiguities, and difficult and acceptable embeddings: the theory has been applied in detail to over 100 types of structures representing these contrasts, with a success rate of about 90%. The account of real-time immediacy includes predictions about the time course of comprehension and a zero-parameter prediction about the average rate of skilled comprehension. Finally, the theory has been successfully applied to a suggestive range of cross-linguistic examples, including constructions from head-final languages such as Japanese.

Many theories have been proposed to explain difficulty with center embedded constructions, most attributing the problem to some kind of limited capacity short-term memory. However, these theories have developed for the most part independently of more traditional memory research, which has focused on uncovering general principles such as chunking and interference. This article attempts to gain some unification with this research by suggesting that an interesting range of core sentence processing phenomena can be explained as interference effects in a sharply limited syntactic working memory. These include difficult and acceptable embeddings, as well as certain limitations on ambiguity resolution, length effects in garden path structures, and the requirement for locality in syntactic structure. The theory takes the form of an architecture for parsing which can index no more than two constituents under the same syntactic relation. A limitation of two or three items shows up in a variety o...

We present a detailed process theory of the moment-by-moment working-memory retrievals and associated control structure that subserve sentence comprehension. The theory is derived from the application of independently motivated principles of memory and cognitive skill to the specialized task of sentence parsing. The resulting theory construes sentence processing as a series of skilled associative memory retrievals modulated by similarity-based interference and fluctuating activation. The cognitive principles are formalized in computational form in the Adaptive Control of Thought–Rational (ACT–R) architecture, and our process model is realized in ACT–R. We present the results of 6 sets of simulations: 5 simulation sets provide quantitative accounts of the effects of length and structural interference on both unambiguous and garden-path structures. A final simulation set provides a graded taxonomy of double center embeddings ranging from relatively easy to extremely difficult. The explanation of center-embedding difficulty is a novel one that derives from the model’s complete reliance on discriminating retrieval cues in the absence of an explicit representation of serial order information. All fits were obtained with only 1 free scaling parameter fixed across the simulations; all other parameters were ACT–R defaults. The modeling results support the hypothesis that fluctuating activation and similarity-based interference are the key factors shaping working memory in sentence processing. We contrast the theory and empirical predictions with several related accounts of sentence-processing complexity.

In human sentence processing, cognitive load can be defined many ways. This report considers a definition of cognitive load in terms of the total probability of structural options that have been disconfirmed at some point in a sentence: the surprisal of word w i given its prefix w 0...i-1 on a phrase-structural language model. These loads can be efficiently calculated using a probabilistic Earley parser (Stolcke, 1995) which is interpreted as generating predictions about reading time on a word-by-word basis. Under grammatical assumptions supported by corpusfrequency data, the operation of Stolcke's probabilistic Earley parser correctly predicts processing phenomena associated with garden path structural ambiguity and with the subject/object relative asymmetry.

Abstract not found

concepts such as weather terms, proper names and emotional terms all segregate into their own meaning spaces. One advantage of representing meaning with vectors such as these is that, since each vector element is a symbol in the input stream (typically another word); all words have as their "features" other words. This translates into the ability to have a vector representation for abstract concepts as easily as one can have a representation for more basic concepts (Burgess & Lund, 1997b). This is important, if not absolutely crucial, when developing a memory model that purports to be general in nature. The other major aspect of categorization that the HAL model can address is the grammatical nature of word meaning. A clear categorization of nouns, prepositions, and Visual inspection of the MDS presentations in this paper all appear to show a robust separation of the various word groups. However, it is important to determine if these categorizations are clearly distinguished in the high-dimensional space. Our approach to this is to use an analysis of variance that compares the intragroup distances to the intergroup distances. This is accomplished by calculating all combinations of item-pair distances within a group and comparing them to all combinations of item-pair distances in the other groups. In all MDS presentations shown in this paper, these analyses were computed, and all differences discussed were reliable. verbs can be seen in Figure 2c. The generalizability of the HAL model to capture grammatical meaning as well as more traditional semantic characteristics of words is an important feature of the model (Burgess, 1998; Burgess & Lund, 1997a) and was part of our motivation to refer to the high-dimensional space as a context space rather than a semantic space. T...

MacWhinney (1978) presented a computational model of the acquisition of morphophonology. The present chapter attempts to extend the model presented in that earlier paper to the acquisition of word-order patterns. This extension is supported by an examination of the previous research on syntactic acquisition. In the final section of the chapter, further possible extensions to phonology and semantics are considered. The crucial claim underlying the basic approach to both morphophonology and syntax is that use of a given rule system is governed by a system of alternative strategies. Within such a multileveled model, alternative strategies can be compared in terms of their relative complexity. In the present chapter, these alternative strategies are evaluated through application of the following analytic technique: 1. A relatively simple strategy that can account for at least some of the observed data is presented. 2. It is shown that there are at least some data that are best explained by this strategy. 3. It is shown that, at some point in development, the child produces forms that cannot be explained by this simple strategy alone. 4. A strategy of somewhat greater complexity and power is introduced and it is shown that this strategy can account for at least some of the data not explained by the simpler (and weaker) strategy. This line of argumentation proceeds until evidence has been presented for six alternative strategies in word-order processing.