Sentence Weighting for Neural Machine Translation Domain Adaptation

Proceedings of the 27th International Conference on Computational Linguistics, pages 3181–3190 Santa Fe, New Mexico, USA, August 20-26, 2018. 3181 Sentence Weighting for Neural Machine Translation Domain Adaptation Shiqi Zhang Deyi Xiong∗ School of Computer Science and Technology, Soochow University, Suzhou, China zzssq77@163.com, dyxiong@suda.edu.cn Abstract In this paper, we propose a new sentence weighting method for the domain adaptation of neural machine translation. We introduce a domain similarity metric to evaluate the relevance between a sentence and an available entire domain dataset. The similarity of each sentence to the target domain is calculated with various methods. The computed similarity is then integrated into the training objective to weight sentences. The adap- tation results on both IWSLT Chinese-English TED task and a task with only synthetic training parallel data show that our sentence weighting method is able to achieve an significant improvement over strong baselines. 1 Introduction Neural machine translation (NMT) has achieved more satisfactory performance than statistical machine translation (SMT) on many language pairs with various advantages over SMT, such as no pipeline-style training, more fluent translations and so on. However, it is still confronted with a big challenge in domain adaptation as the translation quality of NMT is heavily de- pendent on the quantity of training data and the relevance between the training data and the in-domain testing data. The training corpus usually varies across domains, where out-of-domain instances that are relevant to the target domain are beneficial for training while those which are irrelevant to the in-domain may deteriorate translation quality. The widely-used domain adaptation method in NMT is to fine-tune an existing out-of-domain model with the in-domain data (Luong and Manning, 2015). However, the fine-tuning method is of no consideration for the harm caused by irrelevant out-of-domain data. It also tends to overfit rapidly due to the small size of the in-domain data. In this paper, we propose a sentence weighting method that evaluates the weights of sentences with respect to the relevance of sentences to the target domain. We train NMT models by assigning each sentence with a corresponding weight computed according to a domain similarity metric. Domain similarity has been successfully used in some tasks such as parsing, knowledge adaptation, etc.(Plank and Van Noord, 2011; Ruder et al., 2017). We employ domain similarity to measure relevance between the sentences of out-of-domain and in-domain data. In this way, we take into consideration both the goodness and the badness brought by out-of-domain data, in a weighting fashion. Additionally, different from current sentence weighting methods used for NMT that score sentences by making use of other toolkits like the SRI Language Modeling Toolkit (Stolcke, 2002), or an RNN classifier (Chen et al., 2017), our method exploits the information from the NMT system itself. This means that we do not need to train extra toolkits. We also examine the effectiveness of the proposed sentence weighting method on NMT trained with only synthetic parallel data, which is beneficial for the low-resource domain translation. Our method can also be used in …

Content truncated to 500 words