Maastricht University’s Large-Scale Multilingual Machine Translation System for WMT 2021

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 425–430 November 10–11, 2021. ©2021 Association for Computational Linguistics 425 Maastricht University’s Large-Scale Multilingual Machine Translation System for WMT 2021 Danni Liu, Jan Niehues Department of Data Science and Knowledge Engineering, Maastricht University {danni.liu,jan.niehues}@maastrichtuniversity.nl Abstract We present our development of the multilin- gual machine translation system for the large- scale multilingual machine translation task at WMT 2021. Starting form the provided baseline system, we investigated several tech- niques to improve the translation quality on the target subset of languages. We were able to significantly improve the translation quality by adapting the system towards the target subset of languages and by generating synthetic data using the ini- tial model. Techniques successfully applied in zero-shot multilingual machine translation (e.g. similarity regularizer) only had a minor effect on the final translation performance. 1 Introduction This paper describes Maastricht University’s par- ticipation in the large-scale multilingual machine translation task of WMT 2021. We participate in Small Track #2. In this track, the task is to build a translation system between English and 5 South- east Asian languages. The evaluation is performed on all 30 possible translation directions between these languages. We are provided with parallel data extracted from Wikipedia and other sources for all language pairs, as well as a large-scale multilingual machine translation model pretrained on 124 lan- guages (Goyal et al., 2021) including the languages in this task. Starting from the provided baseline models, we investigate several directions in order to improve the performance on the 30 target language direc- tions. As the first step, we focus on methods to adapt the model to these language directions. Specifically, we investigate different strategies to fine-tune the model on the proposed parallel train- ing data. Since the provided parallel data is extremely lim- ited for several translation directions, we investi- gate the use of synthetic parallel data. We focus on jv id ms tl ta en su 44 243 137 440 108 904 jv 644 340 662 46 2,556 id 4,060 2,356 415 48,486 ms 1,174 297 12,023 tl 489 12,348 ta 1,864 Table 1: Number of sentences for each languages pair after preprocessing (in thousand sentences). using pivot languages in order to use well perform- ing language direction to generate training data for worse performing language directions. Finally, we investigate the usefulness of tech- niques to promote the similarity of representation between different languages. While these tech- niques were shown essential for models to perform zero-shot machine translation (Arivazhagan et al., 2019a; Pham et al., 2019; Liu et al., 2021), in our experiments the impact of these methods is only limited. 2 Data We start by introducing the training data and pre- processing steps. 2.1 Languages As required for the small tracks, we only use the data provided by the organizers. The covered lan- guages are: Javanese (jv), Indonesian (id), Malay (ms), Tagalog (tl), Tamil (ta), English (en). Al- though Sundanese (su) is later excluded from the evaluations, we still include it in …

Content truncated to 500 words