Clean data for training statistical MT: the case of MT contamination

Clean Data for Training Statistical MT: The Case of MT Contamination Michel Simard michel.simard@nrc.ca National Research Council Canada Multilingual Text Processing 1200 Montreal Rd., Ottawa, Ontario K1A 0R6 Canada Abstract Users of Statistical Machine Translation (SMT) sometimes turn to the Web to obtain data to train their systems. One problem with this approach is the potential for “MT contamination”: when large amounts of parallel data are collected automatically, there is a risk that a non- negligible portion consists of machine-translated text. Theoretically, using this kind of data to train SMT systems is likely to reinforce the errors committed by other systems, or even by an earlier versions of the same system. In this paper, we study the effect of MT-contaminated training data on SMT quality, by performing controlled simulations under a wide range of conditions. Our experiments highlight situations in which MT contamination can be harmful, and assess the potential of decontamination techniques. 1 Introduction As Statistical Machine Translation (SMT) systems are becoming more widely used in indus- try, we see and hear a growing amount of advice on “best practices” for SMT, most notably regarding system training and data collection and preparation. One recurring theme is that of “clean” data: how training data for SMT systems (which often comes in the form of translation memories) should be exempt of any “dirt”, and how SMT users should go about “cleaning” their data. The most visible proponents of this clean-data approach are probably organizations such as TAUS1, Asia Online2 and PangeaMT3. Multiple forms of “dirt” are said to affect SMT quality. Asia Online (2009) discuss a number of examples: formatting codes and other kinds of markup, such as translation mem- ory metadata; untranslated segments (i.e. translation units in which either the target segment is empty, or a copy of the source segment); segments containing inconsistent or mangled charac- ter encoding. “Badly” translated segments also fall into this category, although it is not quite clear what it means for a segment to be “badly” translated. Some of the examples given by the authors of the Asia Online study suggest that the presence of translator insertions can be considered “bad”, as when, for example, the translator leaves a source-language term in paren- theses for clarity. Non-literal translations are also frowned upon (e.g.: What is a project file? → Pr´esentation des fichiers projet), differing usage of punctuation in the source and target languages, etc. The notion of dirt can extend, in some cases, to out-of-domain data, or even in- domain data containing terminology that is inconsistent with the current task. One translator’s 1https://www.taus.net 2http://www.asiaonline.net 3http://pangeamt.com Al-Onaizan & Simard (Eds.) Proceedings of AMTA 2014, vol. 1: MT Researchers Vancouver, BC © The Authors 69 treasure is another translator’s trash.4 The kinds of problems mentioned in Asia Online’s study are typical of SMT training data extracted from translation memories. Increasingly, however, SMT users are turning to the Web to complement their training sets (Resnik and Smith, 2003). There, other types of dirt can be expected to …

Content truncated to 500 words