The LMU Munich System for the WMT 2021 Large-Scale Multilingual Machine Translation Shared Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 412–417 November 10–11, 2021. ©2021 Association for Computational Linguistics 412 The LMU Munich System for the WMT 2021 Large-Scale Multilingual Machine Translation Shared Task Wen Lai and Jindˇrich Libovický and Alexander Fraser Center for Information and Language Processing, LMU Munich, Germany {lavine, libovicky, fraser}@cis.lmu.de Abstract This paper describes the submission of LMU Munich to the WMT 2021 multilingual ma- chine translation task for small track #1, which studies translation between 6 languages (Croa- tian, Hungarian, Estonian, Serbian, Macedo- nian, English) in 30 directions. We investigate the extent to which bilingual translation sys- tems can influence multilingual translation sys- tems. More specifically, we trained 30 bilin- gual translation systems, covering all language pairs, and used data augmentation techniques such as back-translation and knowledge distil- lation to improve the multilingual translation systems. Our best translation system scores 5 to 6 BLEU higher than a strong baseline sys- tem provided by the organizers (Goyal et al., 2021). As seen in the Dynalab leaderboard, our submission is the only fully constrained submission that uses only the corpus provided by the organizers and does not use any pre- trained models. 1 Introduction Neural Machine Translation (NMT) (Vaswani et al., 2017) has been shown to be effective with rich and in-domain bilingual parallel corpora. Although the NMT model obtained promising performances for high resource language pairs, it is hardly feasible to train translation models for all directions of the lan- guage pairs since the training progress is time- and resource-consuming. Recent work has shown the effectiveness of multilingual neural machine trans- lation (MNMT), which aims to handle the transla- tion from multiple source languages into multiple target languages with a single unified model (John- son et al., 2017; Aharoni et al., 2019; Arivazhagan et al., 2019; Zhang et al., 2020; Fan et al., 2021; Goyal et al., 2021). The MNMT model dramatically reduces train- ing and serving costs. It is faster to train a MNMT model than to train bilingual models for all lan- guage pairs in both directions, and MNMT signif- icantly simplifies deployment in production sys- tems (Johnson et al., 2017; Arivazhagan et al., 2019). Further, parameter sharing across different languages encourages knowledge transfer, which improves low-resource translation directions and potentially enables zero-shot translation (i.e., di- rect translation of a language pair not seen during training) (Ha et al., 2017; Gu et al., 2019; Ji et al., 2020; Zhang et al., 2020). We participate in the WMT 2021 multilingual machine translation task for small track #1. The task aims to train a multilingual model to trans- late 5 Central/East European languages (Croatian, Hungarian, Estonian, Serbian, Macedonian) and English in 30 directions. The multilingual systems presented in this paper are based on the standard paradigm of MNMT proposed by Johnson et al. (2017), which prefixes the source sentence with a special token to indicate the desired target lan- guage and does not change the target sentence at all. Language tags are typically used in MNMT …

Content truncated to 500 words