Abstract

Previous attempts to automatically determine multi-words as the basic unit for language modeling have been successful for extending bigram models [10, 9, 2, 8] to improve the perplexity ofthelanguage model and/or the word accuracy of the speech decoder. However, none ofthese techniques gave improvements over the trigram model so far, except for the rather controlled ATIS task [8]. We therefore propose an algorithm, that minimizes the perplexity improvement ofa bigram model directly. The new algorithm is able to reduce the trigram perplexity andalso achieves word accuracy improvements in the Verbmobil task. It is the natural counterpart of successful word classi cation algorithms for language modeling [4, 7] that minimize the leaving-one-out bigram perplexity. Wealso give some details on the usage of class nding techniques and m-gram models, which can be crucial to successful applications of this technique. 1.

Citations