Adapting Neural Machine Translation with Parallel Synthetic Data

Proceedings of the Conference on Machine Translation (WMT), Volume 1: Research Papers, pages 138–147 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics Adapting Neural Machine Translation with Parallel Synthetic Data Mara Chinea-R´ıos and ´Alvaro Peris and Francisco Casacuberta Pattern Recognition and Human Language Technology Research Center Universitat Polit`ecnica de Val`encia, Val`encia, Spain {machirio, lvapeab, fcn}@prhlt.upv.es Abstract Recent works have shown that the usage of a synthetic parallel corpus can be effec- tively exploited by a neural machine trans- lation system. In this paper, we propose a new method for adapting a general neu- ral machine translation system to a specific task, by exploiting synthetic data. The method consists in selecting, from a large monolingual pool of sentences in the source language, those instances that are more related to a given test set. Next, this selection is automatically translated and the general neural machine translation sys- tem is fine-tuned with these data. For evaluating the adaptation method, we first conducted experiments in two con- trolled domains, with common and well- studied corpora. Then, we evaluated our proposal on a real e-commerce task, yield- ing consistent improvements in terms of translation quality. 1 Introduction Neural machine translation (NMT) (Sutskever et al., 2014; Cho et al., 2014a; Bahdanau et al., 2015) has obtained state-of-the art performance in several domains and language pairs (Sennrich et al., 2016b; Wu et al., 2016). Given the na- ture of NMT paradigms, the limitation for obtain- ing bilingual corpora—or their availability—has been one of the major obstacles faced when build- ing competitive NMT systems. Recently, the idea of using synthetic corpora in NMT has reported promising results with regard to the data scarcity in NMT. Many different works demonstrated that the combination of real parallel corpora with syn- thetic bilingual corpus enhances the NMT trans- lation quality (Sennrich et al., 2016a; Zhang and Zong, 2016a; Cheng et al., 2016). Following these good results, we aim to adapt general NMT models to real, specific tasks by us- ing synthetic parallel data. The core idea is to se- lect the most valuable instances from a large pool of monolingual source sentences, with respect to a given test set. Next, we automatically translate them. Therefore, we obtain a synthetic parallel corpus, related to our test set domain. Such syn- thetic corpus can be used to fine-tune a NMT sys- tem to the domain at hand. The main contributions of this paper involve the necessary steps required to adapt a NMT system to a specific domain: • We propose a novel method to create the most adequate synthetic corpus leverages a vector-space representation of sentences, re- lying on the word embeddings by Mikolov et al. (2013a) and Le and Mikolov (2014). • We describe the pipeline of our adaptation process, relating the selection, translation and fine-tuning processes. • We study our adaptation technique on two classical domains. Additionally, we validate our technique on a real e-commerce transla- tion task. • Results show important improvements over a baseline system. This paper …

Content truncated to 500 words