Multi-Domain Neural Machine Translation

Multi-Domain Neural Machine Translation Sander Tars and Mark Fishel Institute of Computer Science University of Tartu, Estonia tarssander1@gmail.com, fishel@ut.ee Abstract We present an approach to neural ma- chine translation (NMT) that supports mul- tiple domains in a single model and al- lows switching between the domains when translating. The core idea is to treat text domains as distinct languages and use mul- tilingual NMT methods to create multi- domain translation systems; we show that this approach results in significant trans- lation quality gains over fine-tuning. We also explore whether the knowledge of pre- specified text domains is necessary; turns out that it is after all, but also that when it is not known quite high translation quality can be reached, and even higher than with known domains in some cases. 1 Introduction Data-driven machine translation (MT) systems de- pends on the text domain of their training data. In a typical in-domain MT scenario the amount of par- allel texts from a single domain is not enough to train a good translation system, even more so for neural machine translation (NMT; Bahdanau et al., 2014); thus models are commonly trained on a mixture of parallel texts from different domains and then later fine-tuned to in-domain texts (Luong and Manning, 2015). In-domain fine-tuning has two main shortcom- ings: it depends on the availability of sufficient amounts of in-domain data in order to avoid over- fitting and it results in degraded performance for all other domains. The latter means that for trans- c⃝ 2018 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. lating multiple domains one has to run an individ- ual NMT system for each domain. In this work we treat text domains as distinct languages: for example, instead of English-to- Estonian translation we see it as translating En- glish news to Estonian news. We test two mul- tilingual NMT approaches (Johnson et al., 2016; ¨Ostling and Tiedemann, 2017) in a bilingual multi- domain setting and show that both outperform single-domain fine-tuning on all the text domains in our experiments. However, this only works when the text domain is known both when training and translating. In some cases the text domain of the input segment is unknown – for example, web MT systems have to cope with a variety of text domains. Also, some parallel texts do not have a single domain while they are either a mix of texts from different sources (like crawled corpora) or naturally consti- tute a highly heterogeneous mix of texts (like sub- titles or Wikipedia articles). We address these issues by replacing known do- mains with automatically derived ones. At training time we cluster parallel sentences and then apply- ing the multi-domain approach to these clusters. When translating, the input segments are classified as belonging to one of these clusters and translated with this automatically derived information. In the following we review related work in Sec- tion 2, then present our methodology of multi- domain NMT …

Content truncated to 500 words