Are Two Heads Better than One? Crowdsourced Translation via a Two-Step Collaboration of Non-Professional Translators and Editors

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 1134–1144, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Are Two Heads Better than One? Crowdsourced Translation via a Two-Step Collaboration of Non-Professional Translators and Editors Rui Yan, Mingkun Gao, Ellie Pavlick, and Chris Callison-Burch Computer and Information Science Department, University of Pennsylvania, Philadelphia, PA 19104, U.S.A. {ruiyan,gmingkun,epavlick}@seas.upenn.edu, ccb@cis.upenn.edu Abstract Crowdsourcing is a viable mechanism for creating training data for machine trans- lation. It provides a low cost, fast turn- around way of processing large volumes of data. However, when compared to pro- fessional translation, naive collection of translations from non-professionals yields low-quality results. Careful quality con- trol is necessary for crowdsourcing to work well. In this paper, we examine the challenges of a two-step collaboration process with translation and post-editing by non-professionals. We develop graph- based ranking models that automatically select the best output from multiple redun- dant versions of translations and edits, and improves translation quality closer to pro- fessionals. 1 Introduction Statistical machine translation (SMT) systems are trained using bilingual sentence-aligned parallel corpora. Theoretically, SMT can be applied to any language pair, but in practice it produces the state-of-art results only for language pairs with ample training data, like English-Arabic, English- Chinese, French-English, etc. SMT gets stuck in a severe bottleneck for many minority or ‘low resource’ languages with insufficient data. This drastically limits which languages SMT can be successfully applied to. Because of this, collect- ing parallel corpora for minor languages has be- come an interesting research challenge. There are various options for creating training data for new language pairs. Past approaches have examined harvesting translated documents from the web (Resnik and Smith, 2003; Uszkoreit et al., 2010; Smith et al., 2013), or discovering parallel frag- ments from comparable corpora (Munteanu and Marcu, 2005; Abdul-Rauf and Schwenk, 2009; Smith et al., 2010). Until relatively recently, lit- tle consideration has been given to creating par- allel data from scratch. This is because the cost of hiring professional translators is prohibitively high. For instance, Germann (2001) hoped to hire professional translators to create a modest sized 100,000 word Tamil-English parallel corpus, but were stymied by the costs and the difficulty of finding good translators for a short-term commit- ment. Recently, crowdsourcing has opened the possi- bility of translating large amounts of text at low cost using non-professional translators. Facebook localized its web site into different languages us- ing volunteers (TechCrunch, 2008). DuoLingo turns translation into an educational game, and translates web content using its language learners (von Ahn, 2013). Rather than relying on volunteers or gamifica- tion, NLP research into crowdsourcing transla- tion has focused on hiring workers on the Ama- zon Mechanical Turk (MTurk) platform (Callison- Burch, 2009). This setup presents unique chal- lenges, since it typically involves non-professional translators whose language skills are varied, and since it sometimes involves participants who try to cheat to get the small financial reward (Zaidan and Callison-Burch, 2011). A natural approach for trying to …

Content truncated to 500 words