Diving Deep into Context-Aware Neural Machine Translation

Proceedings of the 5th Conference on Machine Translation (WMT), pages 604–616 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 604 Diving Deep into Context-Aware Neural Machine Translation Jingjing Huo1,2 Christian Herold2 Yingbo Gao2 Leonard Dahlmann1 Shahram Khadivi1 Hermann Ney2 1eBay, Inc., Aachen, Germany {jihuo, fdahlmann, skhadivi}@ebay.com 2Human Language Technology and Pattern Recognition Group RWTH Aachen University, Aachen, Germany {surname}@i6.informatik.rwth-aachen.de Abstract Context-aware neural machine translation (NMT) is a promising direction to improve the translation quality by making use of the addi- tional context, e.g., document-level translation, or having meta-information. Although there exist various architectures and analyses, the effectiveness of different context-aware NMT models is not well explored yet. This paper analyzes the performance of document-level NMT models on four diverse domains with a varied amount of parallel document-level bilin- gual data. We conduct a comprehensive set of experiments to investigate the impact of document-level NMT. We find that there is no single best approach to document-level NMT, but rather that different architectures come out on top on different tasks. Looking at task- specific problems, such as pronoun resolution or headline translation, we find improvements in the context-aware systems, even in cases where the corpus-level metrics like BLEU show no significant improvement. We also show that document-level back-translation sig- nificantly helps to compensate for the lack of document-level bi-texts. 1 Introduction Even though machine translation (MT) has greatly improved with the emergence of neural machine translation (NMT) (Sutskever et al., 2014; Bah- danau et al., 2015) and more recently the Trans- former architecture (Vaswani et al., 2017), there remain challenges which can not be solved by us- ing sentence-level NMT systems. Among other issues, this includes the problem of inter-sentential anaphora resolution (Guillou et al., 2018) or the consistent translation across a document (L¨aubli et al., 2018), for which the system inevitably needs document-level context information. In recent years, many works have focused on changing existing NMT architectures to incorpo- rate context information in the translation process (Tiedemann and Scherrer, 2017; Bawden et al., 2018; Voita et al., 2018). However, often times re- sults are reported only on very specific tasks (most commonly subtitle translation), making it difficult to assess the potential of the different methods in a more general setting. This, together with the fact that big improvements are typically reported on low resource tasks, gives the impression that document- level NMT mostly improves due to regularization rather than from leveraging the additional context information. In this work we want to give a more complete overview of the current state of document- level NMT by comparing various approaches on a variety of different tasks including an application- oriented E-commerce setting. We discuss both, widely used performance metrics, as well as highly task-specific observations. Another important aspect when talking about document-level NMT is the applicability in “real life” settings. There, when faced with a low re- source data scenario, back-translation is an es- tablished way of greatly improving system per- formance (Sennrich et al., 2016a). However, to the best of our …

Content truncated to 500 words