Can Automatic Post-Editing Make MT More Meaningful

Can Automatic Post-Editing Make MT More Meaningful? Kristen Parton1 Nizar Habash1 Kathleen McKeown1 Gonzalo Iglesias2 Adri`a de Gispert2 1Columbia University, NY, USA {kristen, kathy, habash}@cs.columbia.edu 2University of Cambridge, Cambridge, UK {gi212, ad465}@eng.cam.ac.uk Abstract Automatic post-editors (APEs) enable the re-use of black box machine translation (MT) systems for a variety of tasks where different aspects of translation are impor- tant. In this paper, we describe APEs that target adequacy errors, a critical problem for tasks such as cross-lingual question-answering, and compare different approaches for post-editing: a rule-based system and a feedback approach that uses a computer in the loop to suggest improve- ments to the MT system. We test the APEs on two different MT systems and across two different genres. Human evaluation shows that the APEs significantly improve adequacy, regardless of approach, MT sys- tem or genre: 30-56% of the post-edited sentences have improved adequacy com- pared to the original MT. 1 Introduction Automatic post-editors (APEs) seek to perform the same task as human post-editors: correcting errors in text produced by machine translation (MT) sys- tems. APEs have been used to target a variety of different types of MT errors, from determiner se- lection (Knight and Chander, 1994) to grammatical agreement (Mareˇcek et al., 2011). There are two main reasons that APEs can improve over decoder output: they can exploit information unavailable to the decoder, and they can carry out deeper text analysis that is too expensive to do in a decoder. We describe APEs that target three types of adequacy errors: deleted content words, content words that were translated into function words, and mistranslated named entities. These types of er- rors are common across statistical MT (SMT) sys- tems and can significantly degrade translation ade- quacy, the amount of information preserved dur- ing translation. Adequacy is critical to the suc- cess of many cross-lingual applications, partic- ularly cross-lingual question answering (CLQA), c⃝ 2012 European Association for Machine Translation. where adequacy errors can significantly decrease task performance. The APEs utilize word align- ments, source- and target-language part-of-speech (POS) tags, and named entities to detect phrase- level errors, and draw on several external resources to find a list of corrections for each error. Once the APEs have a list of errors with pos- sible corrections, we experiment with different ap- proaches to apply the corrections: an approach that uses phrase-level editing rules, and two techniques for passing the corrections as feedback back to the MT systems. The rule-based APE uses word alignments to decide where to insert the top-ranked correction for each error into the target sentence. This approach rewrites the word or phrase where the error was detected, but does not modify the rest of the sentence. We test these MT system- independent rules on two MT systems, MT A and MT B (described in more detail in section ??). The feedback APE passes multiple suggestions for each correction back to the MT system, and allows the MT decoder to determine whether to correct each error and how to correct …

Content truncated to 500 words