We propose that ambiguous prepositional phrase attachment can be resolved on the basis of the relative strength of association of the preposition with noun and verb, estimated on the basis of word distribution in a large corpus. This work suggests that a distributional approach can be effective in resolving parsing problems that apparently call for complex reasoning.

Selectional constraints are limitations on the applicability of predicates to arguments. For example, the statement “The number two is blue” may be syntactically well formed, but at some level it is anomalous — BLUE is not a predicate that can be applied to numbers. According to the influential theory of (Katz and Fodor, 1964), a predicate associates a set of defining features with each argument, expressed within a restricted semantic vocabulary. Despite the persistence of this theory, however, there is widespread agreement about its empirical shortcomings (McCawley, 1968; Fodor, 1977). As an alternative, some critics of the Katz-Fodor theory (e.g. (Johnson-Laird, 1983)) have abandoned the treatment of selectional constraints as semantic, instead treating them as indistinguishable from inferences made on the basis of factual knowledge. This provides a better match for the empirical phenomena, but it opens up a different problem: if selectional constraints are the same as inferences in general, then accounting for them will require a much more complete understanding of knowledge representation and inference than we have at present. The problem, then, is this: how can a theory of selectional constraints be elaborated without first having either an empirically adequate theory of defining features or a comprehensive theory of inference? In this dissertation, I suggest that an answer to this question lies in the representation of conceptual

The problems of access -- retrieving linguistic structure from some mental grammar -- and disambiguation -- choosing among these structures to correctly parse ambiguous linguistic input -- are fundamental to language understanding. The literature abounds with psychological results on lexical access, the access of idioms, syntactic rule access, parsing preferences, syntactic disambiguation, and the processing of garden-path sentences. Unfortunately, it has been difficult to combine models which account for these results to build a general, uniform model of access and disambiguation at the lexical, idiomatic, and syntactic levels. For example psycholinguistic theories of lexical access and idiom access and parsing theories of syntactic rule access have almost no commonality in methodology or coverage of psycholinguistic data. This paper presents a single probabilistic algorithm which models both the access and disambiguation of linguistic knowledge. The algorithm is based on a parallel parser which ranks constructions for access, and interpretations for disambiguation, by their conditional probability. Low-ranked constructions and interpretations are pruned through beam-search; this pruning accounts, among other things, for the garden-path effect. I show that this motivated probabilistic treatment accounts for a wide variety of psycholinguistic results, arguing for a more uniform representation of linguistic knowledge and for the use of probabilisticallyenriched grammars and interpreters as models of human knowledge of and processing of language.

This empirical stady attempts to find answers to the question of how a natural language (heaceforth NL) system could resolve attachment of prepositional phrases (henceforth PPs) by examining natura! ly occurring PP attachments in typed dialogue. Examination includes testing predictive powers of existing attachment theories against the data. The result of this effort will be an algorithm for interpreting PP attachment.

This paper investigates the role of resource allocation as a source of processing difficulty in human sentence comprehension. The paper proposes a simple informationtheoretic characterization of processing difficulty as the work incurred by resource reallocation during parallel, incremental, probabilistic disambiguation in sentence comprehension, and demonstrates its equivalence to the theory of Hale (2001), in which the difficulty of a word is proportional to its surprisal (its negative log-probability) in the context within which it appears. This proposal subsumes and clarifies findings that high-constraint contexts can facilitate lexical processing, and connects these findings to well-known models of parallel constraint-based comprehension. In addition, the theory leads to a number of specific predictions about the role of expectation in syntactic comprehension, including the reversal of locality-based difficulty patterns in syntactically constrained contexts, and conditions under which increased ambiguity facilitates processing. The paper examines a range of established results bearing on these predictions, and shows that they are largely consistent with the surprisal theory.

Abstract not found

The most predominant language processing theories have, for some time, been based largely on structured knowledge and relatively simple rules. These symbolic models intentionally segregate syntactic information processing from statistical information as well as semantic, pragmatic, and discourse influences, thereby minimizing the importance of these potential constraints in learning and processing language. While such models have the advantage of being relatively simple and explicit, they are inadequate to account for learning and validated ambiguity resolution phenomena. In recent years, interactive constraint-based theories of sentence processing have gained increasing support, as a growing body of empirical evidence demonstrates early influences of various factors on comprehension performance. Connectionist networks are one form of model that naturally reflect many properties of constraint-based theories, and thus provide a form in which those theories may be instantiated. Unfortunately, most of the connectionist language models implemented until now have involved severe limitations, restricting the phenomena they could address. Comprehension and production models have, by and large, been limited to simple sentences with small vocabularies (cf. St. John & McClelland, 1990). Most models that have addressed the problem of complex, multi-clausal sentence processing have been prediction networks (cf. Elman, 1991; Christiansen & Chater, 1999a). Although a useful component of a language processing system, prediction does not get at the heart of language: the interface between syntax and semantics.

This article addresses the question of how the processor decides on its initial strategy for syntactic ambiguity resolution. At a point of ambiguity, more than one analysis is possible. An effective strategy might be to adopt the analysis that has most frequently turned out to be correct in the past. Assuming that the world stays the same in most respects, the analysis that has most frequently been correct in the past should provide a good estimate of which analysis is most likely to be correct again. Hence, by adopting this analysis, the processor should make fewer errors than if it chose any other analysis

concepts such as weather terms, proper names and emotional terms all segregate into their own meaning spaces. One advantage of representing meaning with vectors such as these is that, since each vector element is a symbol in the input stream (typically another word); all words have as their "features" other words. This translates into the ability to have a vector representation for abstract concepts as easily as one can have a representation for more basic concepts (Burgess & Lund, 1997b). This is important, if not absolutely crucial, when developing a memory model that purports to be general in nature. The other major aspect of categorization that the HAL model can address is the grammatical nature of word meaning. A clear categorization of nouns, prepositions, and Visual inspection of the MDS presentations in this paper all appear to show a robust separation of the various word groups. However, it is important to determine if these categorizations are clearly distinguished in the high-dimensional space. Our approach to this is to use an analysis of variance that compares the intragroup distances to the intergroup distances. This is accomplished by calculating all combinations of item-pair distances within a group and comparing them to all combinations of item-pair distances in the other groups. In all MDS presentations shown in this paper, these analyses were computed, and all differences discussed were reliable. verbs can be seen in Figure 2c. The generalizability of the HAL model to capture grammatical meaning as well as more traditional semantic characteristics of words is an important feature of the model (Burgess, 1998; Burgess & Lund, 1997a) and was part of our motivation to refer to the high-dimensional space as a context space rather than a semantic space. T...

In lexicalized phrase-structure or dependency parses, a word’s modifiers tend to fall near it in the string. We show that a crude way to use dependency length as a parsing feature can substantially improve parsing speed and accuracy in English and Chinese, with more mixed results on German. We then show similar improvements by imposing hard bounds on dependency length and (additionally) modeling the resulting sequence of parse fragments. This simple “vine grammar ” formalism has only finite-state power, but a context-free parameterization with some extra parameters for stringing fragments together. We exhibit a linear-time chart parsing algorithm with a low grammar constant. 1