Effects of word alignment visualization on post-editing quality & speed

Effects of Word Alignment Visualization on Post-Editing Quality & Speed† Lane Schwartz lanes@illinois.edu Dept. of Linguistics, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA Isabel Lacruz ilacruz@kent.edu Tatyana Bystrova tbystro2@kent.edu Dept. of Modern & Classical Language Studies, Kent State University, Kent, OH 44242, USA Abstract Phrase-based machine translation can be configured to produce alignment data that indicates which machine translated target language words correspond to which original source language words. In most prior work that examined the efficacy of post-editing machine translation, post- editors were presented with machine translations (and in most cases the original source language sentences) without also being presented with source-to-target alignment links. We select four news articles, and ask six Russian-English bilinguals and eleven Spanish-English bilinguals to post-edit English machine translation results, in some cases using alignments and in other cases without. We obtain human adequacy judgements of the post-edited sentences, and demonstrate that when machine translation quality is low, post-editing quality is consistently higher, by a statistically significant amount, when bilingual post-editors are presented with alignment data. 1 Introduction Post-editing is the process whereby a human user corrects the output of a machine translation system. The use of basic post-editing tools by bilingual human translators has been shown to yield substantial increases in terms of productivity (Plitt and Masselot, 2010) as well as improve- ments in translation quality (Green et al., 2013) when compared to bilingual human translators working without assistance from machine translation and post-editing tools. More sophisticated interactive interfaces (Langlais et al., 2000; Barrachina et al., 2009; Koehn, 2009b; Denkowski and Lavie, 2012) may also provide benefit (Koehn, 2009a). The question of how a post-editing interface should be configured and presented to users is a fundamentally interdisciplinary and empirical one. Issues of user interface design, human factors, translation studies, and machine translation quality are all likely relevant. Phrase-based machine translation can be configured to produce alignment data that indicates which machine translated target language words correspond to which original source language words. In most prior work that examined the efficacy of post-editing machine translation, post-editors were presented with machine translations (and in most cases the original source language sentences) without also being presented with source-to-target alignment links. This work begins an attempt to answer two novel questions regarding post-editing interface design: To what extent, if at all, does the presentation of source-to-target word-level alignment links affect the quality or speed of post-editing? Is any such effect, if it exists, dependent on certain aspects of machine translation quality, or on the language pair? †All code, scripts, data & analysis files for this paper are at https://github.com/dowobeha/MT_Summit_2015 Proceedings of MT Summit XV, vol.1: MT Researchers' Track Miami, Oct 30 - Nov 3, 2015 | p. 186 (a) Russian-English adequacy evaluation guidelines 12 The post-edited translation is superior to the reference translation 10 The meaning of the Russian sentence is fully conveyed in the English translation 8 Most of the meaning of the Russian sentence is conveyed in the English translation 6 The English translation …

Content truncated to 500 words