Probabilistic Part-of-Speech Tagging Using Decision Trees (1994)
| Citations: | 412 - 4 self |
BibTeX
@MISC{Schmid94probabilisticpart-of-speech,
author = {Helmut Schmid},
title = {Probabilistic Part-of-Speech Tagging Using Decision Trees},
year = {1994}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
In this paper, a new probabilistic tagging method is presented which avoids problems that Markov Model based taggers face, when they have to estimate transition probabilities from sparse data. In this tagging method, transition probabilities are estimated using a decision tree. Based on this method, a part-of-speech tagger (called TreeTagger) has been implemented which achieves 96.36 % accuracy on Penn-Treebank data which is better than that of a trigram tagger (96.06 %) on the same data. Keywords: Corpus-based NLP, Statistical NLP, Part-of-Speech Tagging. 1 Introduction Word forms are often ambiguous in their part-of-speech (POS). The English word form store for example can be either a noun, a finite verb or an infinitive. In an utterance, this ambiguity is normally resolved by the context of a word: e.g. in the sentence "The 1977 PCs could store two pages of data.", store can only be an infinitive. The predictability of the part-of-speech from the context is used by automatic part-...
