this paper, we study the problem of lexicalizing context-free grammars and show that it enables faster processing. In previous attempts to take advantage of lexicalization, a variety of lexicalization procedures have been developed that convert context-free grammars (CFGs) into equivalent lexicalized grammars. However, these procedures typically suffer from one or more of the following problems

Introduction Many of the formalisms used to define the syntax of natural (and programming) languages may be located in a continuum that ranges from propositional Horn logic to full first order Horn logic, possibly with non-Herbrand interpretations. This structural parenthood has been previously remarked: it lead to the development of Prolog [Col-78, Coh-88] and is analyzed in some detail in [PerW-80] for Context-Free languages and Horn Clauses. A notable outcome is the parsing technique known as Earley deduction [PerW-83]. These formalisms play (at least) three roles: descriptive: they give a finite and organized description of the syntactic structure of the language, analytic: they can be used to analyze sentences so as to retrieve a syntactic structure (i.e. a representation) from which the meaning can be extracted, generative: they can also be used as the specification of the concrete representation of sentences from a more

Stochastic lexicalized context-free grammar (SLCFG) is an attractive compromise between the parsing efficiency of stochastic context-free grammar (SCFG) and the lexical sensitivity of stochastic lexicalized tree-adjoining grammar (SLTAG). SLCFG is a restricted form of SLTAG that can only generate contextfree languages and can be parsed in cubic time. However, SLCFG retains the lexical sensitivity of SLTAG and is therefore a much better basis for capturing distributional information about words than SCFG.

We show how techniques known from generalized LR parsing can be applied to leftcorner parsing. The esulting parsing algorithm for context-free grammars has some advantages over generalized LR parsing: the sizes and generation times of the parsers are smaller, the produced output is more compact, and the basic parsing technique can more easily be adapted to arbitrary context-free grammars.

In this paper a bidirectional parser for Lexicalized Tree Adjoining Grammars will be presented. The algorithm takes advantage of a peculiar characteristic of Lexicalized TAGs, i.e. that each elementary tree is associated with a lexical item, called its anchor. The algorithm employs a mixed strategy: it works bottom -up from the lexical anchors and then expands (.partial) analyses making top-down predictions. Even if such an algorithm does not improve the worst-case time bounds of already known TAGs parsing methods, it could be relevant from the perspective of linguistic information processing, because it employs lexical information in a more direct way.

Abstract not found

We present a new upper bound for the computational complexity of the parsing problem for TAGs, under the constraint that input is read from left to right in a way that errors in the input are observed as soon as possible, which is called the correct-prefix property. The former upper bound was O(n 9 ), which we now improve to O(n 6 ), which is the same as that of practical parsing algorithms for TAGs without the additional constraint of the correct-prefix property. Thereby we show that the correctprefix property does not require significant additional costs. 1 Introduction Traditionally, parsers and recognizers for regular and context-free languages process input from left to right. If a syntax error occurs in the input they often detect that error immediately after its position is reached. The position of the syntax error can be defined as the last input symbol of the shortest prefix which cannot be extended to be a correct sentence in the language L. In formal notation, this p...

Lexicalized context-free grammar (LCFG) is a tree-based formalism that makes use of both tree substitution and a restricted form of tree adjunction. Because of its use of adjunction, LCFG allows sufficient freedom in the way derivations can be performed that lexicalization of context-free grammars (CFGs) is possible while preserving the structure of the trees derived by the CFGs. However, the tree adjunction permitted is sufficiently restricted that LCFGs are string-wise equivalent to CFGs and have the same cubic-time worst-case complexity bounds for recognition and parsing.

Tabular parsers can be defined as deduction systems where formulas, called items, are sets of complete or incomplete constituents (Sikkel, 1997; Shieber, Schabes and Pereira, 1995). Formally, given an input string w = a1... an with n ≥ 0 and a grammar G, a parser IP is a tuple (I, H, D) where I is a set of items, H is a set of hypothesis ([ai, i − 1, i] with 1 ≤ i ≤ n) that encodes the input string, and D is a set of deduction steps that

Bernard LANG INRIA B.P. 105, 78153 Le Chesnay, France lang@inria.inria.fr The intent of this presentation is to exhibit the commonalities between the following syntactic problems: 1. parsing ambiguous or incomplete input, often known as word lattice parsing; 2. parsing ill-formed input, i.e. input that does not belong to the language formally defined by a grammar; 3. syntactic disambiguation of ambiguous sentences; 4. accounting for deviant syntactic structures in grammatical language descriptions. The key idea behind this work is that of a weighted grammar. For simplicity we consider here only a special case of the more general definition given in [Teitelbaum 73], which could lead to other interesting variations (e.g. probabilities as weights, using multiplication instead of addition as below). We define a weighted grammar as a Context-Free (CF) grammar with a numeric weight attached to each of its rules. We attach to any derivation tree a weight that is the sum of the weights of a...