Measuring Confidence Intervals for the Machine Translation Evaluation Metrics (2004)
| Venue: | In Proceedings of the 10th International Conference on Theoretical and Methodological Issues in Machine Translation (TMI-2004 |
| Citations: | 11 - 2 self |
BibTeX
@INPROCEEDINGS{Zhang04measuringconfidence,
author = {Ying Zhang and Stephan Vogel},
title = {Measuring Confidence Intervals for the Machine Translation Evaluation Metrics},
booktitle = {In Proceedings of the 10th International Conference on Theoretical and Methodological Issues in Machine Translation (TMI-2004},
year = {2004},
pages = {4--6}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
Automatic evaluation metrics for Machine Translation (MT) systems, such as BLEU and the related NIST metric, are becoming increasingly important in MT. This paper reports a novel method of calculating the confidence intervals for BLEU/NIST scores using bootstrapping. With this method, we can determine whether two MT systems are significantly different from each other. We study the effect of test set size and number of reference translations on the confidence intervals for these MT evaluation metrics. 1.
