Iterative Domain-Repaired Back-Translation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5884–5893, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 5884 Iterative Domain-Repaired Back-Translation Hao-Ran Wei, Zhirui Zhang, Boxing Chen and Weihua Luo Machine Intelligence Technology Lab Alibaba Group Hangzhou, China {funan.whr, zhirui.zzr, boxing.cbx, weihua.luowh}@alibaba-inc.com Abstract In this paper, we focus on the domain-specific translation with low resources, where in- domain parallel corpora are scarce or nonex- istent. One common and effective strategy for this case is exploiting in-domain mono- lingual data with the back-translation method. However, the synthetic parallel data is very noisy because they are generated by imper- fect out-of-domain systems, resulting in the poor performance of domain adaptation. To address this issue, we propose a novel iter- ative domain-repaired back-translation frame- work, which introduces the Domain-Repair (DR) model to refine translations in synthetic bilingual data. To this end, we construct cor- responding data for the DR model training by round-trip translating the monolingual sen- tences, and then design the unified training framework to optimize paired DR and NMT models jointly. Experiments on adapting NMT models between specific domains and from the general domain to specific domains demon- strate the effectiveness of our proposed ap- proach, achieving 15.79 and 4.47 BLEU im- provements on average over unadapted models and back-translation.1 1 Introduction Neural Machine Translation (NMT) has achieved impressive performance when large amounts of parallel sentences are available (Wu et al., 2016; Vaswani et al., 2017; Hassan et al., 2018). However, some previous works have shown that NMT mod- els perform poorly in specific domains, especially when they are trained on the corpora from very dis- tinct domains (Koehn and Knowles, 2017; Chu and Wang, 2018). The fine-tuning method (Luong and Manning, 2015) is a popular way to mitigate the 1Our code is released in https://github.com/whr94621/ Iterative-Domain-Repaired-Back-Translation effect of domain drift. However, it is not realistic to collect large amounts of high-quality parallel data in every domain we are interested in. Since monolingual in-domain data are usually abundant and easy to obtain, it is essential to explore the un- supervised domain adaptation scenario that utilizes large amounts of out-of-domain bilingual data and in-domain monolingual data. One straightforward and effective solution for unsupervised domain adaptation is to build in- domain synthetic parallel data, including copy- ing monolingual target sentences to the source side (Currey et al., 2017) or back-translation of in-domain monolingual target sentences (Sennrich et al., 2016; Dou et al., 2019). Although the back- translation approach has proven the superior effec- tiveness in exploiting monolingual data, directly applying this method in this scenario brings low- quality in-domain synthetic data. Table 1 gives two incorrect translation sentences generated by back-translation method. The main reason for this situation is that the synthetic parallel data is built by imperfect out-of-domain NMT systems, which leads to inappropriate word expressions or wrong translations. Fine-tuning on such synthetic data is very likely to hurt the performance of domain adaptation. In this paper, we extend back-translation by a Domain-Repair (DR) model to explicitly …

Content truncated to 500 words