Language Branch Gated Multilingual Neural Machine Translation

Proceedings of the 29th International Conference on Computational Linguistics, pages 5046–5053 October 12–17, 2022. 5046 Language Branch Gated Multilingual Neural Machine Translation Haoran Sun and Deyi Xiong∗ College of Intelligence and Computing, Tianjin University, Tianjin, China hrsun0105@gmail.com, dyxiong@tju.edu.cn Abstract Knowledge transfer across languages is crucial for multilingual neural machine translation. In this paper, we propose language branch (LB) gated multilingual neural machine translation that encourages knowledge transfer within the same language branch with a LB-gated mod- ule that is integrated into both the encoder and decoder. The LB-gated module distinguishes LB-specific parameters from global parameters shared by all languages and routes languages from the same LB to the corresponding LB- specific network. Comprehensive experiments on the OPUS-100 dataset show that the pro- posed approach substantially improves transla- tion quality on both middle- and low-resource languages over previous methods. Further anal- ysis demonstrates its ability in learning similar- ities between language branches. 1 Introduction Recent years have witnessed a growing interest in multilingual neural machine translation (NMT), which supports translation among multiple lan- guages with one single model (Dong et al., 2015; Luong et al., 2016; Firat et al., 2016; Johnson et al., 2017; Aharoni et al., 2019; Arivazhagan et al., 2019; Xu et al., 2021). As the parameters of multi- lingual NMT are fully or partially shared by multi- ple languages, knowledge transfer across languages improves translation quality of low-resource lan- guages. Despite these advantages, there still exist chal- lenges in multilingual NMT. As previous studies have found, multilingual NMT usually underper- forms its bilingual counterparts on high-resource languages (Johnson et al., 2017; Arivazhagan et al., 2019). A way to alleviate this issue is to use language-aware modules, which could be more computationally efficient than simply enlarging *Corresponding author. model capacity with deeper/wider models (Zhang et al., 2020, 2021; Lin et al., 2021). As for low- resource languages, positive transfer is more pro- nounced among related languages than distant lan- guages (Sachan and Neubig, 2018). Additionally, explicit language clustering benefits multilingual NMT (Tan et al., 2019) although itself can learn linguistic typology during training (Lu et al., 2018). Inspired by these, we propose language branch (subfamily) gated multilingual neural machine translation, which fuses language branch informa- tion into multilingual NMT by a Language Branch Gated Module (LBGM). A language branch is a subfamily of a language family. Take the Indo- European language family as an example. It can be further divided into subfamilies like Germanic, Slavic, Celtic, etc. The reason why we use lan- guage branches rather than language families is that the latter are relatively coarse-grained. Lan- guages within a language branch are more closely related to each other than those in a language fam- ily. A token that indicates the language branch for the current sentence is fed into LBGM, in addi- tion to the input from other layers in multilingual NMT. With the language branch token, LBGM distinguishes language-branch-specific parameters from global parameters shared by all languages and uses a gate to aggregate these two parts as …

Content truncated to 500 words