This paper investigates the role of resource allocation as a source of processing difficulty in human sentence comprehension. The paper proposes a simple informationtheoretic characterization of processing difficulty as the work incurred by resource reallocation during parallel, incremental, probabilistic disambiguation in sentence comprehension, and demonstrates its equivalence to the theory of Hale (2001), in which the difficulty of a word is proportional to its surprisal (its negative log-probability) in the context within which it appears. This proposal subsumes and clarifies findings that high-constraint contexts can facilitate lexical processing, and connects these findings to well-known models of parallel constraint-based comprehension. In addition, the theory leads to a number of specific predictions about the role of expectation in syntactic comprehension, including the reversal of locality-based difficulty patterns in syntactically constrained contexts, and conditions under which increased ambiguity facilitates processing. The paper examines a range of established results bearing on these predictions, and shows that they are largely consistent with the surprisal theory.

Copyright by Roger Levy 2005 ii

Although proximity between arguments and verbs (locality) is a relatively robust determinant of sentenceprocessing di#culty (Hawkins, 1998, 2001; Gibson, 2000), increasing argument-verb distance can also facilitate processing (Konieczny, 2000). We present two self-paced reading (SPR) experiments involving Hindi that provide further evidence of anti-locality, and a third SPR experiment which suggests that similarity-based interference can attenuate this distance-based facilitation. A unified explanation of interference, locality and anti-locality e#ects is proposed via an independently motivated theory of activation decay and retrieval interference (Anderson et al., 2004). 1

Probabilistic methods are providing new explanatory approaches to fundamental cognitive science questions of how humans structure, process and acquire language. This review examines probabilistic models defined over traditional symbolic structures. Language comprehension and production involve probabilistic inference in such models; and acquisition involves choosing the best model, given innate constraints and linguistic and other input. Probabilistic models can account for the learning and processing of language, while maintaining the sophistication of symbolic models. A recent burgeoning of theoretical developments and online corpus creation has enabled large models to be tested, revealing probabilistic constraints in processing, undermining acquisition arguments based on a perceived poverty

A central question in online human sentence comprehension is: how are linguistic relations established between different parts of a sentence? Previous work has shown that this dependency resolution process can be computationally expensive, but the underlying reasons for this are still unclear. We argue that dependency resolution is mediated by cue-based retrieval, constrained by independently motivated working memory principles defined in a cognitive architecture (ACT-R). To demonstrate this, we investigate an unusual instance of dependency resolution, the processing of negative and positive polarity items, and confirm a surprising prediction of the cue-based retrieval model: partial cue-matches—which constitute a kind of similarity-based interference—can give rise to the intrusion of ungrammatical retrieval candidates, leading to both processing slow-downs and even errors of judgment that take the form of illusions of grammaticality in patently ungrammatical structures. A notable achievement is that good quantitative fits are achieved without adjusting the key model parameters.

Competence-based theories of island effects play a central role in generative grammar, yet the graded nature of many syntactic islands has never been properly accounted for. Categorical syntactic accounts of island effects have persisted in spite of a wealth of data suggesting that island effects are not categorical in nature and that non-structural manipulations that leave island structures intact can radically alter judgments of island violations. We argue here, building on work by Deane, Kluender, and others, that processing factors have the potential to account for this otherwise unexplained variation in acceptability judgments. We report the results of self-paced reading experiments and controlled acceptability studies which explore the relationship between processing costs and judgments of acceptability. In each of the three self-paced reading studies, the data indicate that the processing cost of different types of island violations can be significantly reduced to a degree comparable to that of non-island filler-gap constructions by manipulating a single non-structural factor. Moreover, this reduction in processing cost is accompanied by significant improvements in acceptability. This evidence favors the hypothesis that

Eye fixation durations during normal reading correlate with processing difficulty but the specific cognitive mechanisms reflected in these measures are not well understood. This study finds support in German readers’ eye fixations for two distinct difficulty metrics: surprisal, which reflects the change in probabilities across syntactic analyses as new words are integrated, and retrieval, which quantifies comprehension difficulty in terms of working memory constraints. We examine the predictions of both metrics using a family of dependency parsers indexed by an upper limit on the number of candidate syntactic analyses they retain at successive words. Surprisal models all fixation measures and regression probability. By contrast, retrieval does not model any measure in serial processing. As more candidate analyses are considered in parallel at each word, retrieval can account for the same measures as surprisal. This pattern suggests an important role for ranked parallelism in theories of sentence comprehension.

This is the pre-publication manuscript. The published version may slightly differ.

At least since Kuno & Robinson 1972, it has been widely assumed within generative

We present a series of simulations of behavioral data by casting a simple parsing model in the cognitive architecture ACT-R. We show that constraints defined in ACT-R, specifically those relating to activation, can account for a range of facts about human sentence processing. In doing so, we argue that resource limitation in working memory is better defined as an artefact of very general and independently motivated principles of cognitive processing. 1