Hybrid POS tagging with generalized unknown-word handling (1997)
| Citations: | 8 - 6 self |
BibTeX
@MISC{Lee97hybridpos,
author = {Geunbae Lee and Jeongwon Cha and Jong-hyeok Lee},
title = {Hybrid POS tagging with generalized unknown-word handling},
year = {1997}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
This paper presents POSTAG 1 as a statistical/rulebased hybrid part-of-speech (POS) tagging system with generalized unknown-word handling. The POSTAG integrates morphological analysis with statistical POS disambiguation and post rule-based error-correction. The error-correction rules are automatically learned from a tagged corpus and selectively correct standard HMM tagging errors. The morphological analysis is tightly coupled with the generalized unknown-word handling which uses a morpheme-pattern dictionary that encodes general lexical patterns of Korean morphemes. In this way, we can guess the POS's of unknown-words regardless of their numbers and positions in an eojeol. Experiments demonstrate the effectiveness of our POS tagging methods with generalized unknown-word handling, and the POSTAG will be especially suited to web indexing where most of the indexing terms are unknown words. Keywords: POS tagging, morphological analysis, unknown-word handling, HMM tagging, rule-based ta...
