Sentence-level MT Evaluation Without Reference Translations: Beyond Language Modeling (2005)
| Venue: | In European Association for Machine Translation (EAMT |
| Citations: | 6 - 0 self |
BibTeX
@INPROCEEDINGS{Gamon05sentence-levelmt,
author = {Michael Gamon and Anthony Aue and Martine Smets},
title = {Sentence-level MT Evaluation Without Reference Translations: Beyond Language Modeling},
booktitle = {In European Association for Machine Translation (EAMT},
year = {2005}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
Abstract. In this paper we investigate the possibility of evaluating MT quality and fluency at the sentence level in the absence of reference translations. We measure the correlation between automatically-generated scores and human judgments, and we evaluate the performance of our system when used as a classifier for identifying highly dysfluent and illformed sentences. We show that we can substantially improve on the correlation between language model perplexity scores and human judgment by combining these perplexity scores with class probabilities from a machine-learned classifier. The classifier uses linguistic features and has been trained to distinguish human translations from machine translations. We show that this approach also performs well in identifying dysfluent sentences. 1.
