Crowdsourcing Translation: Professional Quality from Non-Professionals

DMCA

Crowdsourcing Translation: Professional Quality from Non-Professionals

Cached

Download Links

by Omar F. Zaidan , Chris Callison-burch

Citations:

53 - 3 self

BibTeX

@MISC{Zaidan_crowdsourcingtranslation:,
    author = {Omar F. Zaidan and Chris Callison-burch},
    title = {Crowdsourcing Translation: Professional Quality from Non-Professionals},
    year = {}
}

OpenURL

Abstract

Naively collecting translations by crowdsourcing the task to non-professional translators yields disfluent, low-quality results if no quality control is exercised. We demonstrate a variety of mechanisms that increase the translation quality to near professional levels. Specifically, we solicit redundant translations and edits to them, and automatically select the best output among them. We propose a set of features that model both the translations and the translators, such as country of residence, LM perplexity of the translation, edit rate from the other translations, and (optionally) calibration against professional translators. Using these features to score the collected translations, we are able to discriminate between acceptable and unacceptable translations. We recreate the NIST 2009 Urdu-to-English evaluation set with Mechanical Turk, and quantitatively show that our models are able to select translations within the range of quality that we expect from professional translators. The total cost is more than an order of magnitude lower than professional translation. 1

Keyphrases

professional quality professional translator unacceptable translation edit rate professional translation redundant translation collected translation translation quality lm perplexity quality control professional level low-quality result urdu-to-english evaluation mechanical turk total cost non-professional translator yield

DMCA