Letting a Neural Network Decide Which Machine Translation System to Use for Black-Box Fuzzy-Match Repair

Letting a Neural Network Decide Which Machine Translation System to Use for Black-Box Fuzzy-Match Repair John E. Ortega Universitat d’Alacant E-03071, Alacant, Spain jeo10@alu.ua.es Weiyi Lu New York University, 60 5th Avenue New York, New York 10011, USA weiyi.lu@nyu.edu Adam Meyers New York University, 60 5th Avenue New York, New York 10011, USA meyers@cs.nyu.edu Kyunghyun Cho New York University, 60 5th Avenue New York, New York 10011, USA kyunghyun.cho@nyu.edu Abstract While systems using the Neural Network- based Machine Translation (NMT) paradigm achieve the highest scores on recent shared tasks, phrase-based (PBMT) systems, rule-based (RBMT) systems and other systems may get better results for individual examples. Therefore, combined systems should achieve the best results for MT, particularly if the system combi- nation method can take advantage of the strengths of each paradigm. In this paper, we describe a system that predicts whether a NMT, PBMT or RBMT will get the best Spanish translation result for a particular English sentence in DGT-TM 20161. Then we use fuzzy-match repair (FMR) as a mechanism to show that the combined system outperforms individual systems in a black-box machine translation setting. 1 Introduction Natural Language Processing (NLP) systems de- signed to do the same task often belong to different methodological paradigms. At any time in history, the best-scoring systems may tend to come from a particular paradigm. For example, in Machine Translation (MT), the current dominant paradigm is Neural Network-based MT (NMT). The previ- ously dominant paradigm was Phrase Based MT (PBMT), and so on. When comparing MT results for different types of input, systems from certain paradigms perform better on certain types of input c⃝ 2018 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. 1https://ec.europa.eu/jrc/en/language-technologies/dgt- translation-memory and vice versa (Bentivogli et al., 2016). In some cases NMT suffers more than other paradigms (Koehn and Knowles, 2017). Thus, it may be pre- mature to completely abandon “old” methods in favor of “new” ones. Newer methods, especially NMT, tend to achieve higher BLEU scores than previous meth- ods including PBMT and Rule-based MT (RBMT) systems. However, professional translators and users of computer-assisted translation (CAT) tools seem to prefer PBMT output for particular sen- tences (Arenas, 2013). Many recent systems (e.g., participants in WMT17 (Bojar et al., 2017)) use NMT, because it obtains higher scoring results, but does not require time-consuming procedures like feature generation or Quality Estimation (QE) to achieve quality MT translations. CAT tools, and other systems using black-box MT, could benefit from a way of predicting which MT system will perform the best at translating a particular source segment. Such systems which typically use only one MT tool to translate all in- put could benefit from selectively using the output of multiple systems in this way. This paper describes a series of experiments that attempt to take advantage of the strengths of alter- native systems and combine system output to pro- duce the best result. First, we describe our sys- tem, SelecT, which uses a …

Content truncated to 500 words