Domain Adaptation of Document-Level NMT in IWSLT19

Domain Adaptation of Document-Level NMT in IWSLT19 Martin Popel,†‡ Christian Federmann‡ †Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics, Malostranské námˇestí 25, 118 00 Prague, Czech Republic ‡Microsoft, 1 Microsoft Way, Redmond, WA 98121, USA popel@ufal.mff.cuni.cz, chrife@microsoft.com Abstract We describe our four NMT systems submitted to the IWSLT19 shared task in English→Czech text-to-text trans- lation of TED talks. The goal of this study is to understand the interactions between document-level NMT and domain adaptation. All our systems are based on the Transformer model implemented in the Tensor2Tensor framework. Two of the systems serve as baselines, which are not adapted to the TED talks domain: SENTBASE is trained on single sen- tences, DOCBASE on multi-sentence (document-level) se- quences. The other two submitted systems are adapted to TED talks: SENTFINE is fine-tuned on single sentences, DOCFINE is fine-tuned on multi-sentence sequences. We present both automatic-metrics evaluation and manual anal- ysis of the translation quality, focusing on the differences be- tween the four systems. 1. Introduction Neural machine translation (NMT) has recently achieved ex- cellent results in the news translation task. Hassan et al. [1] report achieving a “human parity” on Chinese→English news translation. WMT 2018 overview paper [2, p. 291] re- ports that our English→Czech system “CUNI Transformer” [3] was evaluated as significantly better (p < 0.05) than the human reference. However, it has been shown [4, 5] that evaluating the quality of translation of news articles on iso- lated sentences without the context of the whole document (as done in WMT 2018) is not sufficient. Thus, the research has focused on document-level translation (see e.g. [6, 7, 8]) which is trained simply by training on multi-sentence se- quences.1 Another line of research focuses on domain adaptation of NMT; see [10] for an overview. One of the most simple and effective techniques is fine-tuning [11], where an NMT model trained on (large) general-domain (or “out-domain”) data is further trained on (smaller) in-domain data. The 1Earlier approaches to document-level NMT used more complicated ar- chitectures, e.g. adding a special encoder for encoding the context of previ- ous sentences [9]. term “domain” in domain adaptation is usually understood very broadly – a domain can be defined by any property of the training data (and expected test data), such as the topic, genre, formality, style, written vs. spoken language etc. As far as we know, there is no prior work on the in- teraction of the above-mentioned approaches to NMT: – document-level translation and domain adaptation. Is do- main adaptation of document-level systems different from the domain adaptation of sentence-level systems? What are the differences in the translation output? While we have no definite answers to these questions, we hope our present work brings some new insights into the issue. 2. Systems overview We use the following four systems in our experiments: • SENTBASE is the winning system of the English- Czech WMT 2018 shared task (under name “CUNI Transformer”, i.e. Charles University Transformer). It is described in [3]. It …

Content truncated to 500 words