Statistical language learning research takes the view that many traditional natural language processing tasks can be solved by training probabilistic models of language on a sufficient volume of training data. The design of statistical language learners therefore involves answering two questions: (i) Which of the multitude of possible language models will most accurately reflect the properties necessary to a given task? (ii) What will constitute a sufficient volume of training data? Regarding the first question, though a variety of successful models have been discovered, the space of possible designs remains largely unexplored. Regarding the second, exploration of the design space has so far proceeded without an adequate answer. The goal of this thesis is to advance the exploration of the statistical language learning design space. In pursuit of that goal, the thesis makes two main theoretical contributions: it identifies a new class of designs by providing a novel theory of statistical natural language processing, and it presents the foundations for a predictive theory of data requirements to assist in future design explorations. The first of these contributions is called the meaning distributions theory. This theory

A variety of statistical methods for noun compound analysis are implemented and compared. The results support two main conclusions. First, the use of conceptual association not only enables a broad coverage, but also improves the accuracy. Second, an analysis model based on dependency grammar is substantially more accurate than one based on deepest constituents, even though the latter is more preva- lent in the literature.

The notion of aboutness is fundamental to information retrieval. Assume there is a document d which is about query q. Now, if information is added to d yielding ~ d, the question arises whether document ~ d is about q? In other words, is aboutness monotonic with respect to information composition? This article shows that aboutness does have nonmonotonic character with respect to composition.

Noun compound interpretation is the task of determining the semantic relations among the constituents of a noun compound. For example, “concrete floor” means a floor made of concrete, while “gymnasium floor” is the floor region of a gymnasium. We would like to enable knowledge acquisition systems to interpret noun compounds, as part of their overall task of translating imprecise and incomplete information into formal representations that support automated reasoning. However, if interpreting noun compounds requires detailed knowledge of the constituent nouns, then it may not be worth doing: the cost of acquiring this knowledge may outweigh the potential benefit. This paper describes an empirical investigation of the knowledge required to interpret noun compounds. It concludes that the axioms and ontological distinctions important for this task are derived from the top levels of a hierarchical knowledge base (KB); detailed knowledge of specific nouns is less important. This is good news, not only for our work on knowledge acquisition systems, but also for research on text understanding, where noun compound interpretation has a long history.

. Noun phrases carry much of the information in a text. Systems that attempt to acquire knowledge from text must first decompose complex noun phrases to get access to that information. In the case of noun compounds, this decomposition usually means bracketing the modifiers into nested modifierhead pairs. It is then possible to determine the semantic relationships among individual components of the noun phrase. This paper describes a semi-automatic system for bracketing an unlimited number of adjectival or nominal premodifiers. Since the system is intended to start processing with no prior knowledge, it gets trained as it brackets. That is, it starts from scratch and accumulates bracketing evidence while processing a text under user supervision. Experiments show that generalizations of the structure of complex modifier sequences allow the system to bracket previously unseen compounds correctly. Furthermore, as more compounds are bracketed, the number of bracketing decision...

This paper describes work in progress on part of HAIKU (Delisle et al. 1996), a system to extract semantic information from English technical text. Semantic processing in HAIKU consists of three parts: clause level relationship analysis, case analysis and noun modifier relationship analysis. This paper reports on early work on noun modifier relationship analysis. Research to date includes the construction of a set of semantic labels for the relationships between nouns and their modifiers, the design of algorithms to semi-automatically assign these labels to pairs of elements in noun phrases, as well as an implemented semi-automatic learning bracketer for sequences of multiple premodifiers of head nouns.

The paradigm for nlp known as statistical language learning (sll) has flourished in recent times, being seen as a quick and easy way to get off the ground. Research systems have been launched at many nlp problems including sense disambiguation (Yarowsky, 1992), anaphora resolution (Dagan and Itai, 1990), prepositional phrase attachment (Hindle and Rooth, 1993) and lexical acquisition (Brent, 1993). This has all been fueled by the large text corpora which are increasingly available (Marcus et al., 1993). Since these systems learn to navigate language by consuming text, they are critically dependent on the data that drives them. In this paper I address the practical concern of predicting how much training data is sufficient for a given system. First, I briefly review earlier results and show how these can be combined to bound the expected accuracy of a mode-based learner as a function of the volume of training data. I then develop a more accurate estimate of the expected accuracy function under the assumption that inputs are uniformly distributed. Since this estimate is expensive to compute, I also give a close but cheaply computable approximation to it. Finally, I report on a series of simulations exploring the effects of inputs that are not uniformly distributed.

Introduction Corpus-based, quantitative approaches are important to the study of terminology, because terms are, unlike words, elements which can only be recognised at the level of language fact (Kageura 1995). Despite this, the only work which takes this approach is automatic term recognition (ATR) (Bourigault 1992, Daille/Gaussier/ Lang'e 1994, Enguehard/Pantera 1994, Frantzi/Ananiadou/Tsujii 1996, Justeson/ Katz 1995, Lauriston 1994) and automatic indexing (Salton 1989)[1]. Most of the simple and straightforward quantitative characterisations of terms have already been pursued in ATR work. All ATR methods perform reasonably well, but not completely satisfactorily. In addition, we do not know which method really is better. One reason for this is that ATR work has not clarified its real target. Is it attempting to recognise all the terms in a document, in a corpus, or in a field, or a representative subset? Our standpoint is that the principal target of quantitative terminological s

This paper demonstrates a method for determining the syntactic structure of medical terms. We use a model-fitting method based on the Log Likelihood Ratio to classify three-word medical terms as right or left-branching. We validate this method by computing the agreement between the classification produced by the method and manually annotated classifications. The results show an agreement of 75 %- 83%. This method may be used effectively to enable a wide range of applications that depend on the semantic interpretation of medical terms including automatic mapping of terms to standardized vocabularies and induction of terminologies from unstructured medical text. 1

The notion of aboutness is fundamental to information retrieval. Assume there is a document d which is about query q. Now, if information is added to d yielding ~ d, the question arises whether document ~ d is about q? In other words, is aboutness monotonic with respect to information composition? This article shows that aboutness does have nonmonotonic character with respect to composition. 1 Introduction An often used premise in information retrieval is the following: if a given document d is about the request q, then there is a high likelihood that d will be relevant with respect to the associated information need. Thus, the information retrieval problem is reduced to determining the aboutness relation between documents and requests. Many information retrieval mechanisms have been developed, and there is a wide variation in how they determine aboutness. Recent investigations have centred around formalizing the notion of aboutness by axiomatizing its properties in terms of a neutra...