This paper proposes a new theory of the relationship between the sentence processing mechanism and the available computational resources. This theory -- the Syntactic Prediction Locality Theory (SPLT) -- has two components: an integration cost component and a component for the memory cost associated with keeping track of obligatory syntactic requirements. Memory cost is

Naturally occurring speech contains only a limited amount of complex recursive structure, and this is reflected in the empirically documented difficulties that people experience when processing such structures. We present a connectionist model of human performance in processing recursive language structures. The model is trained on simple artificial languages. We find that the qualitative performance profile of the model matches human behavior, both on the relative difficulty of center-embedded and cross-dependency, and between the processing of these complex recursive structures and right-branching recursive constructions. We analyze how these differences in performance are reflected in the internal representations of the model by performing discriminant analyses on these representation both before and after training. Furthermore, we show how a network trained to process recursive structures can also generate such structures in a probabilistic fashion. This work suggests a novel expla...

Contents 1 Language Complexity in Generative Grammar 3 Part I The Descriptive Complexity of Strongly Context-Free Languages 11 2 Introduction to Part I 13 3 Trees as Elementary Structures 15 4 L 2 K;P and SnS 25 5 Definability and Non-Definability in L 2 K;P 35 6 Conclusion of Part I 57 DRAFT 2 / Contents Part II The Generative Capacity of GB Theories 59 7 Introduction to Part II 61 8 The Fundamental Structures of GB Theories 69 9 GB and Non-definability in L 2 K;P 79 10 Formalizing X-Bar Theory 93 11 The Lexicon, Subcategorization, Theta-theory, and Case Theory 111 12 Binding and Control 119 13 Chains 131 14 Reconstruction 157 15 Limitations of the Interpretation 173 16 Conclusion of Part II 179 A Index of Definitions 183 Bibliography DRAFT 1<

We present a detailed process theory of the moment-by-moment working-memory retrievals and associated control structure that subserve sentence comprehension. The theory is derived from the application of independently motivated principles of memory and cognitive skill to the specialized task of sentence parsing. The resulting theory construes sentence processing as a series of skilled associative memory retrievals modulated by similarity-based interference and fluctuating activation. The cognitive principles are formalized in computational form in the Adaptive Control of Thought–Rational (ACT–R) architecture, and our process model is realized in ACT–R. We present the results of 6 sets of simulations: 5 simulation sets provide quantitative accounts of the effects of length and structural interference on both unambiguous and garden-path structures. A final simulation set provides a graded taxonomy of double center embeddings ranging from relatively easy to extremely difficult. The explanation of center-embedding difficulty is a novel one that derives from the model’s complete reliance on discriminating retrieval cues in the absence of an explicit representation of serial order information. All fits were obtained with only 1 free scaling parameter fixed across the simulations; all other parameters were ACT–R defaults. The modeling results support the hypothesis that fluctuating activation and similarity-based interference are the key factors shaping working memory in sentence processing. We contrast the theory and empirical predictions with several related accounts of sentence-processing complexity.

We introduce a novel parser based on a probabilistic version of a left-corner parser. The left-corner strategy is attractive because rule probabilities can be conditioned on both top-down goals and bottom-up derivations. We develop the underlying theory and explain how a grammar can be induced from analyzed data. We show that the left-corner approach provides an advantage over simple top-down probabilistic context-free grammars in parsing the Wall Street Journal using a grammar induced from the Penn Treebank. We also conclude that the Penn Treebank provides a fairly weak testbed due to the flatness of its bracketings and to the obvious overgeneration and undergeneration of its induced grammar.

In this paper we develop novel algorithmic ideas for building a natural language parser grounded upon the hypothesis of incrementality. Although widely accepted and experimentally supported under a cognitive perspective as a model of the human parser, the incrementality assumption has never been exploited for building automatic parsers of unconstrained real texts. The essentials of the hypothesis are that words are processed in a left-to-right fashion, and the syntactic structure is kept totally connected at each step. Our proposal relies on a machine learning technique for predicting the correctness of partial syntactic structures that are built during the parsing process. A recursive neural network architecture is employed for computing predictions after a training phase on examples drawn from a corpus of parsed sentences, the Penn Treebank. Our results indicate the viability of the approach and lay out the premises for a novel generation of algorithms for natural language processing which more closely model human parsing. These algorithms may prove very useful in the development of ecient parsers. 1

Abstract. This paper shows how proof nets can be used to formalize the notion of “incomplete dependency ” used in psycholinguistic theories of the unacceptability of center-embedded constructions. Such theories of human language processing can usually be restated in terms of geometrical constraints on proof nets. The paper ends with a discussion of the relationship between these constraints and incremental semantic interpretation. 1.

One of the central problems in the study of human language processing is ambiguity resolution: How do people resolve the extremely pervasive ambiguity of the language they encounter? One possible answer to this question is suggested by experiencebased models, which claim that people typically resolve ambiguities in a way which has been successful in the past. In order to determine the course of action that has been "successful in the past" when faced with some ambiguity, it is necessary to generalise over past experience. In this paper, we will present a computational experience-based model, which learns to generalise over linguistic experience from exposure to syntactic structures in a corpus. The model is a hybrid system, which uses symbolic grammars to build and represent syntactic structures, and neural networks to rank these structures on the basis of its experience. We use a dynamic grammar, which provides a very tight correspondence between grammatical derivations and incremental processing, and recursive neural networks, which are able to deal with the complex hierarchical structures produced by the grammar. We demonstrate that the model reproduces a number of the structural preferences found in the experimental psycholinguistics literature, and also performs well on unrestricted text.

ing away from irrelevant details is a theoretical virtue, but the kinds of abstractions that module geography makes can lead to incorrect inferences from data. That such a possibility exists is clearly demonstrated by the working memory research of Just & Carpenter (1992). Briefly, Just and Carpenter have argued that some garden path effects that were previously interpreted in terms of a syntactically encapsulated module can instead be explained by individual differences in working memory capacity. Such an explanation is not considered in a theoretical framework that systematically ignores the role of memory structures in parsing. This point should be taken regardless of whether one is convinced by the current body of empirical support for this particular model---the fact remains that such an explanation could in principle account for the data, and these alternative explanations are only discovered by developing functionally complete architectures. The next few sections describes what ...

Abstract. This paper presents a novel method for wide coverage parsing using an incremental strategy, which is psycholinguistically motivated. A recursive neural network is trained on treebank data to learn first pass attachments, and is employed as a heuristic for guidingparsingdecision. The parser is lexically blind and uses beam search to explore the space of plausible partial parses and returns the full analysis havinghighest probability. Results are based on preliminary tests on the WSJ section of the Penn treebank and suggest that our incremental strategy is a computationally viable approach to parsing. 1