Can Domains Be Transferred across Languages in Multi-Domain Multilingual Neural Machine Translation?

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 381 - 396 December 7-8, 2022 ©2022 Association for Computational Linguistics Can Domains Be Transferred Across Languages in Multi-Domain Multilingual Neural Machine Translation? Thuy-Trang Vu♢∗and Shahram Khadivi† Xuanli He♢ and Dinh Phung♢ and Gholamreza Haffari♢ ♢Department of Data Science and AI, Monash University, Australia † eBay Inc. {trang.vu1,xuanli.he1,first.last}@monash.edu skhadivi@ebay.com Abstract Previous works mostly focus on either multi- lingual or multi-domain aspects of neural ma- chine translation (NMT). This paper investi- gates whether the domain information can be transferred across languages on the composi- tion of multi-domain and multilingual NMT, particularly for the incomplete data condition where in-domain bitext is missing for some language pairs. Our results in the curated leave- one-domain-out experiments show that multi- domain multilingual (MDML) NMT can boost zero-shot translation performance up to +10 gains on BLEU, as well as aid the generali- sation of multi-domain NMT to the missing domain. We also explore strategies for ef- fective integration of multilingual and multi- domain NMT, including language and domain tag combination and auxiliary task training. We find that learning domain-aware representations and adding target-language tags to the encoder leads to effective MDML-NMT. 1 Introduction Multilingual NMT (MNMT), which enables a sin- gle model to support translation across multiple directions, has attracted a lot of interest both in the research community and industry. The gap be- tween MNMT and bilingual counterparts has been reduced significantly, and even for some settings, it has been shown to surpass bilingual NMT (Tran et al., 2021). MNMT enables knowledge sharing among languages, and reduces model training, de- ployment, and maintenance costs. On the other hand, multi-domain NMT aims to build robust NMT models, providing high-quality translation on diverse domains. While multilingual and multi- domain NMT are highly appealing in practice, they are often studied separately. To accommodate the domain aspect, previous MNMT works focus on learning a domain-specific ∗Work done while doing internship at eBay Inc. Figure 1: An example of the multi-domain multilingual incomplete data condition (best seen in colours). (a) The colour indicates the availability of bitext in the corresponding domain for each language. (b) Domain and language-pair matrix for the data condition in (a). MNMT by finetuning a general NMT model on the domain of interest (Tran et al., 2021; Bérard et al., 2020). Recently, Cooper Stickland et al. (2021) pro- pose to unify multilingual and multi-domain NMT into a holistic system by stacking language-specific and domain-specific adapters with a two-phase training process. Thanks to the plug-and-play abil- ity of adapters, their system can handle translation across multiple languages and support multiple do- mains. However, as each domain adapter is learned independently, their adapter-based model lacks the ability of effective knowledge sharing among do- mains. In this paper, we take a step further toward uni- fying multilingual and multi-domain NMT into a single setting and model, i.e., multi-domain multi- lingual NMT (MDML-NMT), and enable effective knowledge sharing across both domains and lan- guages. Unlike the complete data assumption in …

Content truncated to 500 words