Back-translation for Large-Scale Multilingual Machine Translation

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 418–424 November 10–11, 2021. ©2021 Association for Computational Linguistics 418 Back-translation for Large-Scale Multilingual Machine Translation Baohao Liao Shahram Khadivi Sanjika Hewavitharana eBay Inc. {baliao|skhadivi|shewavitharana}@ebay.com Abstract This paper illustrates our approach to the shared task on large-scale multilingual ma- chine translation in the sixth conference on machine translation (WMT-21). In this work, we aim to build a single multilingual trans- lation system with a hypothesis that a uni- versal cross-language representation leads to better multilingual translation performance. We extend the exploration of different back- translation methods from bilingual translation to multilingual translation. Better perfor- mance is obtained by the constrained sam- pling method, which is different from the find- ing of the bilingual translation. Besides, we also explore the effect of vocabularies and the amount of synthetic data. Surprisingly, the smaller size of vocabularies perform better, and the extensive monolingual English data offers a modest improvement. We submit- ted to both the small tasks and achieve the second place. The code and trained models are available at https://github.com/ BaohaoLiao/multiback. 1 Introduction Bilingual neural machine translation (NMT) sys- tems have achieved decent performance with the help of Transformer (Vaswani et al., 2017). One of the most exciting recent trends in NMT is training a single system on multiple languages at once (John- son et al., 2017b; Aharoni et al., 2019a; Zhang et al., 2020; Fan et al., 2020). This is a power- ful paradigm for two reasons: simplifying sys- tem development and deployment, and improving the translation quality on low-resource language pairs by transferring similar knowledge from high- resource languages. This paper describes our experiments on the task of large-scale multilingual machine translation in WMT-21. We primarily focus on the small tasks, es- pecially on Small Task 2 which has a small amount of training data. Small Task 1 contains five Cen- tral/East European languages and English, having 30 translation directions. Similarly, Small Task 2 contains five South East Asian languages and English, also having 30 translation directions. In this work, we mainly concentrate on different back-translation methods (Sennrich et al., 2016a; Edunov et al., 2018; Graça et al., 2019) for multi- lingual machine translation, including beam search and other sampling methods. Along with it, we also explore the effect of different sizes of vocabularies and the effect of various amounts of synthetic data. On this large-scale multilingual machine transla- tion task, we achieved the second place for both small tasks, obtaining 34.96 and 33.34 average sp- BLEU scores (Goyal et al., 2021) on the hidden test set for the Small Task 1 and 2, respectively. 2 Related Work Multilingual Neural Machine Translation has received increasing attention recently. Since Dong et al. (2015) extended the traditional bilingual NMT to one-to-many translation, there has been a mas- sive increase in work on MT systems that involve more than two languages (Dabre et al., 2017; Choi et al., 2018; Chu and Dabre, 2019). The recent research on multilingual NMT can be split into …

Content truncated to 500 words