Statistical Phrase-Based Post-Editing

Proceedings of NAACL HLT 2007, pages 508–515, Rochester, NY, April 2007. c⃝2007 Association for Computational Linguistics Statistical Phrase-based Post-editing Michel Simard Cyril Goutte Interactive Language Technologies National Research Council of Canada Gatineau, Canada, K1A 0R6 FirstName.LastName@nrc.gc.ca Pierre Isabelle Abstract We propose to use a statistical phrase- based machine translation system in a post-editing task: the system takes as in- put raw machine translation output (from a commercial rule-based MT system), and produces post-edited target-language text. We report on experiments that were per- formed on data collected in precisely such a setting: pairs of raw MT output and their manually post-edited versions. In our evaluation, the output of our automatic post-editing (APE) system is not only bet- ter quality than the rule-based MT (both in terms of the BLEU and TER metrics), it is also better than the output of a state- of-the-art phrase-based MT system used in standalone translation mode. These re- sults indicate that automatic post-editing constitutes a simple and efficient way of combining rule-based and statistical MT technologies. 1 Introduction The quality of machine translation (MT) is gener- ally considered insufficient for use in the field with- out a significant amount of human correction. In the translation world, the term post-editing is often used to refer to the process of manually correcting MT output. While the conventional wisdom is that post- editing MT is usually not cost-efficient compared to full human translation, there appear to be situations where it is appropriate and even profitable. Unfortu- nately, there are few reports in the literature about such experiences (but see Allen (2004) for exam- ples). One of the characteristics of the post-editing task, as opposed to the revision of human translation for example, is its partly repetitive nature. Most MT systems invariably produce the same output when confronted with the same input; in particular, this means that they tend to make the same mistakes over and over again, which the post-editors must correct repeatedly. Batch corrections are sometimes pos- sible when multiple occurrences of the same mis- take appear in the same document, but when it is repeated over several documents, or equivalently, when the output of the same machine translation system is handled by multiple post-editors, then the opportunities for factoring corrections become much more complex. MT users typically try to reduce the post-editing load by customizing their MT sys- tems. However, in Rule-based Machine Translation (RBMT), which still constitutes the bulk of the cur- rent commercial offering, customization is usually restricted to the development of “user dictionaries”. Not only is this time-consuming and expensive, it can only fix a subset of the MT system’s problems. The advent of Statistical Machine Translation, and most recently phrase-based approaches (PBMT, see Marcu and Wong (2002), Koehn et al. (2003)) into the commercial arena seems to hold the promise of a solution to this problem: because the MT sys- tem learns directly from existing translations, it can be automatically customized to new domains and tasks. However, the success of this operation cru- …

Content truncated to 500 words