Generalised Unsupervised Domain Adaptation of Neural Machine Translation with Cross-Lingual Data Selection

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3335–3346 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3335 Generalised Unsupervised Domain Adaptation of Neural Machine Translation with Cross-Lingual Data Selection Thuy-Trang Vu Xuanli He Dinh Phung Gholamreza Haffari Department of Data Science and AI Faculty of Information Technology, Monash University, Australia {trang.vuthithuy,xuanli.he1,first.last}@monash.edu Abstract This paper considers the unsupervised domain adaptation problem for neural machine trans- lation (NMT), where we assume the access to only monolingual text in either the source or target language in the new domain. We propose a cross-lingual data selection method to extract in-domain sentences in the missing language side from a large generic monolin- gual corpus. Our proposed method trains an adaptive layer on top of multilingual BERT by contrastive learning to align the represen- tation between the source and target language. This then enables the transferability of the domain classifier between the languages in a zero-shot manner. Once the in-domain data is detected by the classifier, the NMT model is then adapted to the new domain by jointly learning translation and domain discrimination tasks. We evaluate our cross-lingual data se- lection method on NMT across five diverse domains in three language pairs, as well as a real-world scenario of translation for COVID- 19. The results show that our proposed method outperforms other selection baselines up to +1.5 BLEU score. 1 Introduction Unsupervised domain adaptation (UDA) aims to generalise MT models trained on domains with typ- ically large-scale bilingual parallel text to new do- mains without parallel data (Chu and Wang, 2018). Most prior works in UDA of NMT assume the availability of either non-parallel texts of both lan- guages or only the target-language monolingual text in the new domain to adapt the NMT model. The adaptation is achieved by modifying the model architecture and joint training with other auxiliary tasks (Gulcehre et al., 2015; Domhan and Hieber, 2017; Dou et al., 2019), or constructing a parallel corpus for the new domain from a general-domain parallel text using data-selection methods (Silva et al., 2018; Hu et al., 2019). However, very little attention has been paid to the UDA problem with only the source-language monolingual text in the new domain. In practice, this setting is not very rare, e.g. building a translation system from En- glish to Shona (a low-resource African language) in a specific domain such as healthcare and disaster. While it would be very time consuming to collect in-domain text in Shona, English corpora are more accessible. In this paper, we consider the generalised prob- lem of UDA in NMT where we assume the avail- ability of monotext in only one language, either the source or target, in the new domain. We propose a generalised approach to the problem using cross- lingual data selection to extract sentences in the new domain for the missing language side from a large monolingual generic corpus. Our proposed data selection method trains an adaptive layer on top of multilingual BERT by contrastive learning (Chen …

Content truncated to 500 words