Context-Aware Monolingual Repair for Neural Machine Translation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 877–886, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 877 Context-Aware Monolingual Repair for Neural Machine Translation Elena Voita1,2 Rico Sennrich4,3 Ivan Titov3,2 1Yandex, Russia 2University of Amsterdam, Netherlands 3University of Edinburgh, Scotland 4University of Zurich, Switzerland lena-voita@yandex-team.ru sennrich@cl.uzh.ch ititov@inf.ed.ac.uk Abstract Modern sentence-level NMT systems often produce plausible translations of isolated sen- tences. However, when put in context, these translations may end up being inconsistent with each other. We propose a monolingual DocRepair model to correct inconsistencies between sentence-level translations. DocRe- pair performs automatic post-editing on a se- quence of sentence-level translations, refining translations of sentences in context of each other. For training, the DocRepair model re- quires only monolingual document-level data in the target language. It is trained as a monolingual sequence-to-sequence model that maps inconsistent groups of sentences into consistent ones. The consistent groups come from the original training data; the inconsis- tent groups are obtained by sampling round- trip translations for each isolated sentence. We show that this approach successfully imi- tates inconsistencies we aim to fix: using con- trastive evaluation, we show large improve- ments in the translation of several contex- tual phenomena in an English→Russian trans- lation task, as well as improvements in the BLEU score. We also conduct a human eval- uation and show a strong preference of the annotators to corrected translations over the baseline ones. Moreover, we analyze which discourse phenomena are hard to capture us- ing monolingual data only.1 1 Introduction Machine translation has made remarkable progress, and studies claiming it to reach a human parity are starting to appear (Hassan et al., 2018). However, when evaluating translations of the whole documents rather than isolated sentences, human raters show a stronger preference for 1The code and data sets (including round-trip translations) are available at https://github.com/lena-voita/ good-translation-wrong-in-context. human over machine translation (Läubli et al., 2018). These findings emphasize the need to shift towards context-aware machine translation both from modeling and evaluation perspective. Most previous work on context-aware NMT as- sumed that either all the bilingual data is avail- able at the document level (Jean et al., 2017; Wang et al., 2017; Tiedemann and Scherrer, 2017; Baw- den et al., 2018; Voita et al., 2018; Maruf and Haffari, 2018; Agrawal et al., 2018; Kuang et al., 2018; Miculicich et al., 2018) or at least its frac- tion (Voita et al., 2019). But in practical scenarios, document-level parallel data is often scarce, which is one of the challenges when building a context- aware system. We introduce an approach to context-aware machine translation using only monolingual document-level data. In our setting, a sep- arate monolingual sequence-to-sequence model (DocRepair) is used to correct sentence-level translations of adjacent sentences. The key idea is to use monolingual data to imitate typical incon- sistencies between context-agnostic translations of isolated sentences. The DocRepair model is trained to map inconsistent groups of sentences into consistent …

Content truncated to 500 words