A Survey of Domain Adaptation for Neural Machine Translation

Proceedings of the 27th International Conference on Computational Linguistics, pages 1304–1319 Santa Fe, New Mexico, USA, August 20-26, 2018. 1304 A Survey of Domain Adaptation for Neural Machine Translation Chenhui Chu Institute for Datability Science Osaka University chu@ids.osaka-u.ac.jp Rui Wang National Institute of Information and Communications Technology wangrui@nict.go.jp Abstract Neural machine translation (NMT) is a deep learning based approach for machine translation, which yields the state-of-the-art translation performance in scenarios where large-scale parallel corpora are available. Although the high-quality and domain-specific translation is crucial in the real world, domain-specific corpora are usually scarce or nonexistent, and thus vanilla NMT performs poorly in such scenarios. Domain adaptation that leverages both out-of-domain parallel corpora as well as monolingual corpora for in-domain translation, is very important for domain- specific translation. In this paper, we give a comprehensive survey of the state-of-the-art domain adaptation techniques for NMT. 1 Introduction Neural machine translation (NMT) (Cho et al., 2014; Sutskever et al., 2014; Bahdanau et al., 2015) allows for end-to-end training of a translation system without the need to deal with word alignments, translation rules and complicated decoding algorithms, which are characteristics of statistical machine translation (SMT) systems (Koehn et al., 2007). NMT yields the state-of-the-art translation performance in resource rich scenarios (Bojar et al., 2017; Nakazawa et al., 2017). However, currently, high quality parallel corpora of sufficient size are only available for a few language pairs such as languages paired with English and several European language pairs. Furthermore, for each language pair the sizes of the domain specific corpora and the number of domains available are limited. As such, for the majority of language pairs and domains, only few or no parallel corpora are available. It has been known that both vanilla SMT and NMT perform poorly for domain specific translation in low resource scenarios (Duh et al., 2013; Sennrich et al., 2013; Zoph et al., 2016; Koehn and Knowles, 2017). High quality domain specific machine translation (MT) systems are in high demand whereas general purpose MT has limited applications. In addition, general purpose translation systems usually perform poorly and hence it is important to develop translation systems for specific domains (Koehn and Knowles, 2017). Leveraging out-of-domain parallel corpora and in-domain monolingual corpora to improve in- domain translation is known as domain adaptation for MT (Wang et al., 2016; Chu et al., 2018). For example, the Chinese-English patent domain parallel corpus has 1M sentence pairs (Goto et al., 2013), but for the spoken language domain parallel corpus there are only 200k sentences available (Cettolo et al., 2015). MT typically performs poorly in a resource poor or domain mismatching scenario and thus it is important to leverage the spoken language domain data with the patent domain data (Chu et al., 2017). Furthermore, there are monolingual corpora containing millions of sentences for the spoken language domain, which can also be leveraged (Sennrich et al., 2016b). There are many studies of domain adaptation for SMT, which can be mainly divided into two cat- egories: data centric and model …

Content truncated to 500 words