An Empirical Investigation of Discounting in Cross-Domain Language Models

by Greg Durrett , Dan Klein

Datum	Value	Source
TITLE	An Empirical Investigation of Discounting in Cross-Domain Language Models	SVM HeaderParse 0.2
AUTHOR NAME	Greg Durrett	SVM HeaderParse 0.2
AUTHOR AFFIL	Computer Science Division; University of California, Berkeley	SVM HeaderParse 0.2
AUTHOR NAME	Dan Klein	SVM HeaderParse 0.2
AUTHOR AFFIL	Computer Science Division; University of California, Berkeley	SVM HeaderParse 0.2
ABSTRACT	We investigate the empirical behavior of n-gram discounts within and across domains. When a language model is trained and evaluated on two corpora from exactly the same domain, discounts are roughly constant, matching the assumptions of modified Kneser-Ney LMs. However, when training and test corpora diverge, the empirical discount grows essentially as a linear function of the n-gram count. We adapt a Kneser-Ney language model to incorporate such growing discounts, resulting in perplexity improvements over modified Kneser-Ney and Jelinek-Mercer baselines. 1	SVM HeaderParse 0.2
CITATIONS	12 found	ParsCit 1.0