Selectional constraints are limitations on the applicability of predicates to arguments. For example, the statement “The number two is blue” may be syntactically well formed, but at some level it is anomalous — BLUE is not a predicate that can be applied to numbers. According to the influential theory of (Katz and Fodor, 1964), a predicate associates a set of defining features with each argument, expressed within a restricted semantic vocabulary. Despite the persistence of this theory, however, there is widespread agreement about its empirical shortcomings (McCawley, 1968; Fodor, 1977). As an alternative, some critics of the Katz-Fodor theory (e.g. (Johnson-Laird, 1983)) have abandoned the treatment of selectional constraints as semantic, instead treating them as indistinguishable from inferences made on the basis of factual knowledge. This provides a better match for the empirical phenomena, but it opens up a different problem: if selectional constraints are the same as inferences in general, then accounting for them will require a much more complete understanding of knowledge representation and inference than we have at present. The problem, then, is this: how can a theory of selectional constraints be elaborated without first having either an empirically adequate theory of defining features or a comprehensive theory of inference? In this dissertation, I suggest that an answer to this question lies in the representation of conceptual

Statistical language learning research takes the view that many traditional natural language processing tasks can be solved by training probabilistic models of language on a sufficient volume of training data. The design of statistical language learners therefore involves answering two questions: (i) Which of the multitude of possible language models will most accurately reflect the properties necessary to a given task? (ii) What will constitute a sufficient volume of training data? Regarding the first question, though a variety of successful models have been discovered, the space of possible designs remains largely unexplored. Regarding the second, exploration of the design space has so far proceeded without an adequate answer. The goal of this thesis is to advance the exploration of the statistical language learning design space. In pursuit of that goal, the thesis makes two main theoretical contributions: it identifies a new class of designs by providing a novel theory of statistical natural language processing, and it presents the foundations for a predictive theory of data requirements to assist in future design explorations. The first of these contributions is called the meaning distributions theory. This theory

Conceptual natural language processing systems usually rely on case frame instantiation to-recognize events and role objects in text. But generating a good set of case frames for a domain is timeconsuming, tedious, and prone to errors of omission. We have developed a corpus-based algorithm for acquiring conceptual case frames empirically from unannotated text. Our algorithm builds on previous research on corpus-based methods for acquiring extraction patterns and semantic lexicons. Giv. en extraction patterns and a semantic lexicon for a domain, our algorithm learns semantic preferences for each extraction pattern and merges the syntactically compatible patterns to produce multi-slot case frames with selectional restrictions. The case frames generate more cohesive output and produce fewer false hits than the original extraction patterns. Our system requires only proclassified training texts and a few hours of manual review to filter the dictionar- ies, demonstrating that conceptual case frames can be acquired from unannotated text without special training resources.

this paper, as well as utilizing such new and promising resources as on-line dictionaries. Recent work on machine-readable dictionaries offers new and interesting possibilities both for the computerassisted lexicology (see Walker 1984) and for construct- ing lexical databases derived from the definitions in machine-readable dictionaries and utilized in NLP along with other fields (see Amsler 1982, 1984a; Walker, Amsler 1986, Calzolari 1984a,b). The premises and goals of these efforts are fully compatible with our belief, first, that no AI system is ready to make the kind of decisions that lexicon building requires and, second, that 'simply having an online version of an encyclopedia [or a dictionary] would be of little use, as there is practically nothing that current AI could draw from the raw text. Rather, we must carefully re-represent the encyclopedia's knowledge -- by hand -- into some more structured form' (Lenat et al. 1986:75). Such re-representation would be necessary for Amsler's (1984a:458) 'lexical knowledge base [which] is a repos- itory of computational information about concepts' and which contains information derived from machine-read- able dictionaries, the full text of reference books, the results of statistical analysis of text usages, and data manually obtained from human world knowledge.' This paper deals primarily with the last item on Amsler's agenda. It is based on the following approach to the problem of lexicon building. The work is done by humans assisted by an interactive aid which enhances productivity and ensures uniformity. It is important to recognize that lexicon building in NLP involves the acquisition of not one entity but rather of three interre- lated but distinct lexicons, namely a) the world concept lexicon which structures our kno...

Machine-readable versions of everyday dictionaries have been seen as a likely source of information for use in natural language processing because they contain an enormous amount of lexical and semantic knowledge. However, after 15 years of research, the results appear to be disappointing. No comprehensive evaluation of machine-readable dictionaries (MRDs) as a knowledge source has been made to date, although this is necessary to determine what, if anything, can be gained from MRD research. To this end, this paper will first consider the postulates upon which MRD research has been based over the past fifteen years, discuss the validity of these postulates, and evaluate the results of this work. We will then propose possible future directions and applications that may exploit these years of effort, in the light of current directions in not only NLP research, but also fields such as lexicography and electronic publishing.

Many conventional approaches to text analysis and information retrieval prove ineffective when large text collections must be processed in heterogeneous subject areas. An alternative text manipulation system is outlined useful for the retrieval of large heterogeneous texts, and for the recognition of content similarities between text excerpts, based on flexible text matching procedures carried out in several contexts of different scope. The methods are illustrated by search experiments performed with the 29-volume Funk and Wagnalls encyclopedia.

The \Naive Physics Manifesto " of Pat Hayes (1978) proposes a large-scale project of developing a formal theory encompassing the entire knowledge of physics of naive reasoners, expressed in a declarative symbolic form. The theory is organized in clusters of closely interconnected concepts and axioms. More recent work in the representation of commonsense physical knowledge has followed a somewhat di erent methodology. The goal has been to develop a competence theory powerful enough to justify commonsense physical inferences, and the research is organized in microworlds, each microworld covering a small range of physical phenomena. In this paper we compare the advantages and disadvantages of the two approaches. Three Scenarios Consider the following scenario: Common sense is a wild thing, savage, and beyond rules.

Access-Limited Logic (ALL) is a language for knowledge representation which formalizes the access limitations inherent in a network structured knowledge-base. Where a deductive method such as resolution would retrieve all assertions that satisfy a given pattern, an access-limited logic retrieves all assertions reachable by following an available access path. The time complexity of inference is thus a polynomial function of the size of the accessible portion of the knowledge-base, rather than the size of the entire knowledge-base. Access-Limited Logic, though incomplete, still has a well defined semantics and a weakened form of completeness, Socratic Completeness, which guarantees that for any query which is a logical consequence of the knowledge-base, there exists a series of queries after which the original query will succeed. We have implemented ALL in Lisp and it has been used to build several non-trivial systems, including versions of Qualitative Process Theory and Pearl's probability networks. ALL is a step toward providing the properties-- clean semantics, efficient inference, expressive power-- which will be necessary to build large, effective knowledge

This paper considers the composition of tuples from two relations in order to derive additional tuples of one of these relations. Our purpose is to determine when the composition is plausible and for which relation the new tuples are derived. We first present a formal definition of composition and our extension to it. We next define conditions on the domains and ranges of the relations that are necessary for extended composition to occur. We then show how a set of underlying attributes, independently specified for each relation, is sufficient for determining plausible composition, when the primitives are combined according to an algebra. Finally, we apply our method for extended composition to a representative group of semantic relations and evaluate the results. 1

We have employed Algorithmic Probability Theory to construct a system for machine learning of great power and generality. The principal thrust of present research is the design of sequences of problems to train this system. Current programs for machine learning are limited in the kinds of concepts accessible to them, the kinds of problems they can learn to solve, and in the efficiency with which they learn — both in computation time needed and/or in amount of data needed for learning. Algorithmic Probability Theory provides a general model of the learning process that enables us to understand and surpass many of these limitations. Starting with a machine containing a small set of concepts, we use a carefully designed sequence of problems of increasing difficulty to bring the machine to a high level of problem solving skill. The use of training sequences of problems for machine knowledge acquisition promises to yield Expert Systems that will be easier to train and free of the brittleness that characterizes the narrow specialization of present day systems of this sort. It is also expected that the present research will give needed insight in the design of training sequences for human learning.