The IICT-Yverdon System for the WMT 2021 Unsupervised MT and Very Low Resource Supervised MT Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 973–981 November 10–11, 2021. ©2021 Association for Computational Linguistics 973 The IICT-Yverdon System for the WMT 2021 Unsupervised MT and Very Low Resource Supervised MT Task Àlex R. Atrio1,2 and Gabriel Luthier1 and Axel Fahy1 and Giorgos Vernikos1,2 and Andrei Popescu-Belis1,2 and Ljiljana Dolamic3 1HEIG-VD / HES-SO 2EPFL 3Armasuisse, W+T Yverdon-les-Bains Lausanne Thun Switzerland Switzerland Switzerland name.surname@heig-vd.ch ljiljana.dolamic@armasuisse.ch Abstract In this paper, we present the systems submit- ted by our team from the Institute of ICT (HEIG-VD / HES-SO) to the Unsupervised MT and Very Low Resource Supervised MT task. We first study the improvements brought to a baseline system by techniques such as back-translation and initialization from a par- ent model. We find that both techniques are beneficial and suffice to reach performance that compares with more sophisticated sys- tems from the 2020 task. We then present the application of this system to the 2021 task for low-resource supervised Upper Sorbian (HSB) to German translation, in both directions. Fi- nally, we present a contrastive system for HSB- DE in both directions, and for unsupervised German to Lower Sorbian (DSB) translation, which uses multi-task training with various training schedules to improve over the base- line. 1 Introduction In this paper, we present the systems submitted to the WMT 2021 task on Unsupervised MT and Very Low Resource Supervised MT. We first build a series of baseline systems, driven mostly by con- siderations of simplicity, trained on data from the 2020 edition of the task, for translation between Upper Sorbian (HSB) and German (DE). These sys- tems, described in Section 3, enable us to quantify the merits of using additional back-translated data (Sennrich et al., 2016) and of initializing the sys- tem for a low-resource pair with parameters learned on a high-resource pair (same target language and related source language). The systems described above serve as the ba- sis for our 2021 baseline submitted to the shared task, for DE→HSB and HSB→DE, presented in Section 4, which improves upon our 2020 base- line with the addition of more parallel data, and achieves competitive performance with the use of back-translation and parent-initialization only. However, this approach does not lead to an effec- tive baseline for unsupervised German to Lower Sorbian (DSB) translation (Section 5). In Section 6, we present experiments with a contrastive system that implements multi-task learning, with several schedules, in which denoising tasks together with translation are presented to the systems in increas- ing order of complexity, leading to more robust HSB↔DE systems, together with a strategy of di- verse ensembling. We also use our DE→HSB sys- tem to initialize a multi-task DE→DSB system for the unsupervised task, although in this case the performance is not competitive. 2 Datasets We use various Upper Sorbian datasets from the 2020 edition of the task, and additional WMT data, as presented in Table 1. The monolingual HSB data from 2020 comes from three sources: sorbian_institute_monolingual con- sists of a mix of high- and …

Content truncated to 500 words