Modeling Morphologically Rich Languages Using Split Words and Unstructured Dependencies
by
Deniz Yuret
,
Ergun Biçici
BibTeX
@MISC{Yuret_modelingmorphologically,
author = {Deniz Yuret and Ergun Biçici},
title = {Modeling Morphologically Rich Languages Using Split Words and Unstructured Dependencies},
year = {}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
We experiment with splitting words into their stem and suffix components for modeling morphologically rich languages. We show that using a morphological analyzer and disambiguator results in a significant perplexity reduction in Turkish. We present flexible n-gram models, Flex-Grams, which assume that the n−1 tokens that determine the probability of a given token can be chosen anywhere in the sentence rather than the preceding n − 1 positions. Our final model achieves 27 % perplexity reduction compared to the standard n-gram model. 1
