This paper investigates the impact of Constraint Dependency Grammars (CDG) on the accuracy of an integrated speech recognition and CDG parsing system. We compare a conventional CDG with CDGs that are induced from annotated sentences and template-expanded sentences. The grammars are evaluated on parsing speed, precision/coverage, and improvement of word and sentence accuracy of the integrated system. Sentence-derived CDGs significantly improve recognition accuracy over the conventional CDG but are less general. Expanding the sentences with templates provides us with a mechanism for increasing the coverage of the grammar with only minor reductions in recognition accuracy.

In this thesis we develop a discriminative learning method for dependency parsing using online large-margin training combined with spanning tree inference algorithms. We will show that this method provides state-of-the-art accuracy, is extensible through the feature set and can be implemented efficiently. Furthermore, we display the language independent nature of the method by evaluating it on over a dozen diverse languages as well as show its practical applicability through integration into a sentence compression system. We start by presenting an online large-margin learning framework that is a generaliza- tion of the work of Crammer and Singer [34, 37] to structured outputs, such as sequences and parse trees. This will lead to the heart of this thesis – discriminative dependency pars- ing. Here we will formulate dependency parsing in a spanning tree framework, yielding efficient parsing algorithms for both projective and non-projective tree structures. We will then extend the parsing algorithm to incorporate features over larger substructures with- out an increase in computational complexity for the projective case. Unfortunately, the non-projective problem then becomes NP-hard so we provide structurally motivated ap- proximate algorithms. Having defined a set of parsing algorithms, we will also define a rich feature set and train various parsers using the online large-margin learning framework. We then compare our trained dependency parsers to other state-of-the-art parsers on 14 diverse languages: Arabic, Bulgarian, Chinese, Czech, Danish, Dutch, English, German, Japanese, Portuguese, Slovene, Spanish, Swedish and Turkish. Having built an efficient and accurate discriminative dependency parser, this thesis will then turn to improving and applying the parser. First we will show how additional re- sources can provide useful features to increase parsing accuracy and to adapt parsers to new domains. We will also argue that the robustness of discriminative inference-based learning algorithms lend themselves well to dependency parsing when feature representa- tions or structural constraints do not allow for tractable parsing algorithms. Finally, we integrate our parsing models into a state-of-the-art sentence compression system to show its applicability to a real world problem.

The eliminative nature of Constraint Satisfaction over finite domains offers an interesting potential for robustness in the parsing of spoken language. An approach is presented, which puts unusually ambitious demands on the design of the Constraint Satisfaction procedure by trying to combine preferential reasoning, dynamic scheduling, parallel processing and incremental constraint solving within a coherent solution. 1 Introduction Spoken language processing involves a number of novel requirements as compared to traditional techniques for the treatment of written text. Among those which deserve special attention are: 1. the frequent occurrence of deviant language in particular if utterances are produced spontaneously as part of a problem solving task. Spoken language is not only characterized by hesitations, breaks and restarts, but is additionally marked by different kinds of syntactic and semantic inconsistencies. Most surprisingly, these performance phenomena hardly disturb the la...

Constraint Dependency Grammar (CDG) is a constraint-based grammatical formalism which has a weak generative capacity beyond context-free grammars and supports a very flexible parsing algorithm for working with feature grammars; however, the running time of the parser is O(n 4 ). Hence, we have investigated how to improve the running time of the parser by applying feature constraints differentially and by using aggregate unary constraints, which can be applied in O(n 2 ) time. Additional speedup was achieved by integrating the filtering algorithm more tightly with the parser and by supporting the use of a varying number of roles for word classes. Key words: Parsing, NLP, Constraint Satisfaction. Introduction: CDG Parsing Constraint Dependency Grammar, introduced by Maruyama [1, 2, 3], uses constraints to determine which dependencies are grammatical for a sentence. The parsing algorithm is framed as a constraint satisfaction problem: the parsing rules are the constraints and the s...

ROLE VALUES A Thesis Submitted to the Faculty Purdue University by Christopher M. White In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy May 2000 - ii - To my loving wife Margit.

Despite a long and venerable tradition in descriptive linguistics, dependency grammar has until recently played a fairly marginal role both in theoretical linguistics and in natural language processing. The increasing interest in dependency-based

In this paper an application of guided local search (GLS) to the problem of natural language parsing is presented. The given parsing approach is situated in a constraint based parsing paradigm [10] that allows natural language processing in a robust and resource adaptive way [16]. Some extensions of GLS are introduced, most notably a multi-threaded search where a couple of agents cooperate with each other in parallel, showing synergetic effects. The resulting algorithm is compared to competing techniques within the framework of weighted constraint dependency grammars [21]. An experimental evaluation shows GLS being on par with similar approaches [7].

We investigate the task of performance prediction for language models belonging to the exponential family. First, we attempt to empirically discover a formula for predicting test set cross-entropy for n-gram language models. We build models over varying domains, data set sizes, and n-gram orders, and perform linear regression to see whether we can model test set performance as a simple function of training set performance and various model statistics. Remarkably, we find a simple relationship that predicts test set performance with a correlation of 0.9997. We analyze why this relationship holds and show that it holds for other exponential language models as well, including class-based models and minimum discrimination information models. Finally, we discuss how this relationship can be applied to improve language model performance. 1

In this paper, we describe a prototype spoken language system that loosely integrates a speech recognition component based on hidden Markov models with a constraint dependency grammar (CDG) parser using a word graph to pass sentence candidates between the two modules. This loosely coupled system was able to improve the sentence selection accuracy and concept accuracy over the level achieved by the acoustic module with a stochastic grammar. Timing profiles suggest that a tighter coupling of the modules could reduce parsing times of the system, as could the development of better acoustic models and tighter parsing constraints for conjunctions. 1. INTRODUCTION In this paper, we describe a prototype of a spoken language system that integrates a speech recognition component based on hidden Markov models with a constraint dependency grammar (CDG) parser. The underlying goal of our combined system is to identify the 'best' overall sentence candidate with respect to all available knowledge s...

MUSE CSP (MU ltiply SEgmented C onstraint Satisfaction Problem) [1, 2] is an extension to the constraint satisfaction problem (CSP), which is especially useful for problems that segment into multiple instances of CSP that share variables. In Helzerman and Harper [2], the concepts of MUSE node, arc, and path consistency were defined and algorithms for MUSE arc consistency, MUSE AC-1, and MUSE path consistency were developed. MUSE AC-1 is similar to the CSP arc consistency algorithm AC-4 [3]. Recently, Bessi`ere developed a new algorithm, AC-6 [4], which has the same worst-case running time as AC-4 but is faster than AC-3 and AC-4 in practice. In this paper, we focus on developing two faster MUSE arc consistency algorithms: MUSE AC-2, which directly applies Bessi`ere's method to improve upon MUSE AC-1; and MUSE AC-3, which uses a fast evaluation method for managing the additional sets required by the MUSE approach. These new algorithms decrease the number of steps required to achieve arc...