We present a detailed process theory of the moment-by-moment working-memory retrievals and associated control structure that subserve sentence comprehension. The theory is derived from the application of independently motivated principles of memory and cognitive skill to the specialized task of sentence parsing. The resulting theory construes sentence processing as a series of skilled associative memory retrievals modulated by similarity-based interference and fluctuating activation. The cognitive principles are formalized in computational form in the Adaptive Control of Thought–Rational (ACT–R) architecture, and our process model is realized in ACT–R. We present the results of 6 sets of simulations: 5 simulation sets provide quantitative accounts of the effects of length and structural interference on both unambiguous and garden-path structures. A final simulation set provides a graded taxonomy of double center embeddings ranging from relatively easy to extremely difficult. The explanation of center-embedding difficulty is a novel one that derives from the model’s complete reliance on discriminating retrieval cues in the absence of an explicit representation of serial order information. All fits were obtained with only 1 free scaling parameter fixed across the simulations; all other parameters were ACT–R defaults. The modeling results support the hypothesis that fluctuating activation and similarity-based interference are the key factors shaping working memory in sentence processing. We contrast the theory and empirical predictions with several related accounts of sentence-processing complexity.

This paper investigates the role of resource allocation as a source of processing difficulty in human sentence comprehension. The paper proposes a simple informationtheoretic characterization of processing difficulty as the work incurred by resource reallocation during parallel, incremental, probabilistic disambiguation in sentence comprehension, and demonstrates its equivalence to the theory of Hale (2001), in which the difficulty of a word is proportional to its surprisal (its negative log-probability) in the context within which it appears. This proposal subsumes and clarifies findings that high-constraint contexts can facilitate lexical processing, and connects these findings to well-known models of parallel constraint-based comprehension. In addition, the theory leads to a number of specific predictions about the role of expectation in syntactic comprehension, including the reversal of locality-based difficulty patterns in syntactically constrained contexts, and conditions under which increased ambiguity facilitates processing. The paper examines a range of established results bearing on these predictions, and shows that they are largely consistent with the surprisal theory.

Abstract not found

Given the, recent evidence for t)robabilist;ic lnechanisms in models of hmnan ambiguity resolution, this paper investigates the plausibility of exl)loiting current wide-coverage, 1)rob- abilistic parsing techniques to model hmnan linguistic pertbrmance. In particular, we investigate the tmrtbrlnance of standard stoolrestic parsers when they are revised to operate incrementally, and with reduced memory resources. We present techniques fi)r ranking and filtering malyses together with experilnen- tal results. Our results confirm that stochas- tic parsers which adhere to these 1)sychologically motivated constraints achieve good pertYrlmmce. Memory can be reduced down to l% (compared to exhausitve sero:oh) without re- ducing recall and precision. Additionally, these models exhibit subsl, antially fisi;cr l)erfi)rmance. Finally, we argue that this general resull, is likely to hold fi)r more sophisticated, and i)sycholinguistically plausible, probalfilistic parsing 1nod- els.

Copyright by Roger Levy 2005 ii

A central question for psycholinguistics concerns the role of grammatical constraints in online sentence processing. Many current theories maintain that the language processing mechanism constructs a parse or parses that are grammatically consistent with the whole of the perceived input each time it processes a word. Several bottom-up, dynamical models make a contrasting prediction: partial parses which are syntactically compatible with only a proper subpart of the input are sometimes constructed, at least temporarily. Three self-paced reading experiments probed for interference from such locally coherent structures. The first tested for a distracting effect of irrelevant Subject–Predicate interpretations of Noun Phrase–Verb Phrase sequences (e.g., The coach smiled at the player tossed a frisbee) on reading times. The second addressed the question of whether the interference effects can be treated as lexical interference, instead of involving the formation of locally coherent syntactic structures. The third replicated the reading time effects of the first two experiments with grammaticality judgments. We evaluate the dynamical account, comparing it to other approaches that also predict effects of local coherence, and arguing against accounts which rule out the formation of merely locally coherent structures.

One of the central problems in the study of human language processing is ambiguity resolution: How do people resolve the extremely pervasive ambiguity of the language they encounter? One possible answer to this question is suggested by experiencebased models, which claim that people typically resolve ambiguities in a way which has been successful in the past. In order to determine the course of action that has been "successful in the past" when faced with some ambiguity, it is necessary to generalise over past experience. In this paper, we will present a computational experience-based model, which learns to generalise over linguistic experience from exposure to syntactic structures in a corpus. The model is a hybrid system, which uses symbolic grammars to build and represent syntactic structures, and neural networks to rank these structures on the basis of its experience. We use a dynamic grammar, which provides a very tight correspondence between grammatical derivations and incremental processing, and recursive neural networks, which are able to deal with the complex hierarchical structures produced by the grammar. We demonstrate that the model reproduces a number of the structural preferences found in the experimental psycholinguistics literature, and also performs well on unrestricted text.

A fundamental goal of psycholinguistic research is to understand the architectures and mechanisms that underlie language comprehension. Such an account entails an understanding of the representation and organization of linguistic knowledge in the mind and a theory of how that knowledge is used dynamically to recover the interpretation of the utterances we encounter. While research in theoretical and computational linguistics has demonstrated the tremendous complexities of language understanding, our intuitive experience of language is rather different. For the most part people understand the utterances they encounter effortlessly and accurately. In constructing models of how people comprehend language, we are thus presented with what we dub the performance paradox: How is it that people understand language so effectively given such complexity and ambiguity? In our pursuit and evaluation of new theories, we typically consider how well a particular model is able to account for observed results from the relevant range of controlled psycholinguistic experiments (empirical adequacy), and also the ability of the model to explain why the language comprehension system has the form and function it does (explanatory adequacy). Interestingly, research over the past twenty-five years has led to tremendous variety in proposals for parsing, disambiguation, and reanalysis mechanisms, many of which have been realized as computational models. However, while it is possible to classify models – e.g., according to whether they are modular, interactive, serial, parallel, or probabilistic – consensus at any concrete level has been largely

Abstract not found

An incremental dependency parser’s probability model is entered as a predictor in a linear mixed-effects model of German readers’ eye-fixation durations. This dependencybased predictor improves a baseline that takes into account word length, n-gram probability, and Cloze predictability that are typically applied in models of human reading. This improvement obtains even when the dependency parser explores a tiny fraction of its search space, as suggested by narrow-beam accounts of human sentence processing such as Garden Path theory.