Deterministic Part-of-Speech Tagging with Finite-State Transducers

DMCA

Deterministic Part-of-Speech Tagging with Finite-State Transducers (1995)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Emmanuel Roche , Yves Schabes

Venue:	Computational Linguistics
Citations:	96 - 0 self

BibTeX

@ARTICLE{Roche95deterministicpart-of-speech,
    author = {Emmanuel Roche and Yves Schabes},
    title = {Deterministic Part-of-Speech Tagging with Finite-State Transducers},
    journal = {Computational Linguistics},
    year = {1995},
    volume = {21},
    pages = {227--253}
}

OpenURL

Abstract

Stochastic approaches to natural language processing have often been preferred to rule-based approaches because of their robustness and their automatic training capabilities. This was the case for part-of-speech tagging until Brill showed how state-of-the-art part-of-speech tagging can be achieved with a rule-based tagger by inferring rules from a training corpus. However, current implementations of the rule-based tagger run more slowly than previous approaches. In this paper, we present a finite-state tagger, inspired by the rule-based tagger, that operates in optimal time in the sense that the time to assign tags to a sentence corresponds to the time required to follow a single path in a deterministic finite-state machine. This result is achieved by encoding the application of the rules found in the tagger as a nondeterministic finite-state transducer and then turning it into a deterministic transducer. The resulting deterministic transducer yields a part-of-speech tagger whose speed is dominated by the access time of mass storage devices. We then generalize the techniques to the class of transformation-based systems. 1.

Keyphrases

deterministic part-of-speech tagging rule-based tagger finite-state transducer optimal time nondeterministic finite-state transducer state-of-the-art part-of-speech tagging deterministic transducer single path deterministic finite-state machine part-of-speech tagging mass storage device natural language processing transformation-based system stochastic approach rule-based approach automatic training capability current implementation access time deterministic transducer yield previous approach part-of-speech tagger finite-state tagger training corpus

DMCA