Abstract

In statistical natural language processing we always face the problem of sparse data. One way to reduce this problem is to group words into equivalence classes which is a standard method in statistical language modeling. In this paper we describe a method to determine bilingual word classes suitable for statistical ma- chine translation. We develop an opti- mization criterion based on a maximum- likelihood approach and describe a clustering algorithm. We will show that the usage of the bilingual word classes we get can improve statistical machine transla- tion.

Citations

540	Class-based n-gram models of natural language - Brown, Pietra, et al. - 1992
245	The alignment template approach to statistical machine translation - Och, Ney - 2004
155	HMM-based word alignment in statistical translation - Vogel, Ney, et al. - 1996
90	Threshold accepting: a general purpose optimization algorithm appearing superior to simulated annealing - Dueck, Scheuer - 1990
82	Improved clustering techniques for class-based statistical language modelling - Kneser, Ney - 1993
50	Finite-state speech-to-speech translation - Vidal - 1997
17	Forming word classes by statistical clustering for statistical language modelling - Kneser, Ney - 1993
11	Improving statistical natural language translation with categories and rules - Och, Weber - 1998
10	Automatic word classification using simulated annealing - Jardino, Adda - 1993
4	HMM-based Word Alignment In Statistical Translation - Wang, Collins, et al. - 1996
2	Coerced markov models for cross-lingual lexical-tag relations - Fung, Wu - 1995