Exploring Domain-shared and Domain-specific Knowledge in Multi-Domain Neural Machine Translation

Proceedings of Machine Translation Summit XIX, Vol. 1: Research Track, pages 99–110 September 4–8, 2023, Macau SAR, China. ©2023 The authors. This article is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). 99 Exploring Domain-shared and Domain-specific Knowledge in Multi-Domain Neural Machine Translation Zhibo Man zhiboman@bjtu.edu.cn Yujie Zhang∗ yjzhang@bjtu.edu.cn Yuanmeng Chen yuanmengchen@bjtu.edu.cn Yufeng Chen yfchen@bjtu.edu.cn Jinan Xu jaxu@bjtu.edu.cn School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China Abstract Currently, multi-domain neural machine translation (NMT) has become a significant research topic in domain adaptation machine translation, which trains a single model by mixing data from multiple domains. Multi-domain NMT aims to improve the performance of the low- resources domain through data augmentation. However, mixed domain data brings more trans- lation ambiguity. Previous work focused on domain-general or domain-context knowledge learning, respectively. Therefore, there is a challenge for acquiring domain-general or domain- context knowledge simultaneously. To this end, we propose a unified framework for learning simultaneously domain-general and domain-specific knowledge, we are the first to apply pa- rameter differentiation in multi-domain NMT. Specifically, we design the differentiation crite- rion and differentiation granularity to obtain domain-specific parameters. Experimental results on multi-domain UM-corpus English-to-Chinese and OPUS German-to-English datasets show that the average BLEU scores of the proposed method exceed the strong baseline by 1.22 and 1.87, respectively. In addition, we investigate the case study to illustrate the effectiveness of the proposed method in acquiring domain knowledge. 1 Introduction In recent years, Neural Machine Translation (NMT) has shown excellent performance in various translation tasks, as evidenced by state-of-the-art (SOTA) results reported in studies such as (Bahdanau et al., 2015; Wu et al., 2016; Vaswani et al., 2017; Liu et al., 2021; Fernandes et al., 2022), among which Multi-domain NMT aims to construct a single NMT model with the ability to translate sentences across different domains (Wang et al., 2020). Mixed-domain data can improve cross-domain knowledge on low-resource domains by data augmentation. However, word ambiguity increases when we mix data from multiple domains. Therefore, a challenge remains in how to learn the domain-shared and domain-specific knowledge for multi-domain NMT. To address the above problem, researchers design domain-shared (Zeng et al., 2018, 2019; Pham et al., 2019; Wang et al., 2020) and domain-specific knowledge learning mechanisms 100 Example 1 : Translation based on domain-shared knowledge learning Input (Law) promotion centers and technology enterprise incubation base Reference 促进中心和科技企业孵化✓基地 Mixed 促进中心和技术企业孵化✓基地 Single 促进中心和技术企业潜伏×基地 MDNMT (Jiang et al., 2020) 促进中心和技术企业培养×基地 Mixed data (Education) incubation (孵化✓) lasts anywhere from 24-28 days. Example 2 : Translation based on domain-specific knowledge learning Input (Science) output power can never equal the input power for there always losses. Reference 输出功率决不可能等于输入功率✓因为总有损耗。 Mixed 输出电力×从来不等于输入电力×由于常有损耗。 Single 输出功率从来不等于输入功率✓因为总有损耗。 MDNMT (Jiang et al., 2020) 输出功率从不等于输入功率✓由于总有损耗。 Mixed data (News) which is also the source of most of Beijing‘s power (电力×) supply. Table 1: Two English-Chinese translations of “incubation” and “power” with different models. (Kobus et al., 2016; Britz et al., 2017; Jiang et al., 2020; Lee et al., 2022). Nevertheless, these strategies have inherent limitations, …

Content truncated to 500 words