The University of Helsinki and Aalto University submissions to the WMT 2020 news and low-resource translation tasks

Proceedings of the 5th Conference on Machine Translation (WMT), pages 1129–1138 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 1129 The University of Helsinki and Aalto University submissions to the WMT 2020 news and low-resource translation tasks Yves Scherrer University of Helsinki yves.scherrer@helsinki.fi Stig-Arne Gr¨onroos Aalto University stig-arne.gronroos@aalto.fi Sami Virpioja University of Helsinki sami.virpioja@helsinki.fi Abstract This paper describes the joint participation of University of Helsinki and Aalto University to two shared tasks of WMT 2020: the news translation between Inuktitut and English and the low-resource translation between German and Upper Sorbian. For both tasks, our ef- forts concentrate on efficient use of monolin- gual and related bilingual corpora with sched- uled multi-task learning as well as an opti- mized subword segmentation with sampling. Our submission obtained the highest score for Upper Sorbian → German and was ranked sec- ond for German → Upper Sorbian according to BLEU scores. For English–Inuktitut, we reached ranks 8 and 10 out of 11 according to BLEU scores. 1 Introduction Our work is motivated by Gr¨onroos et al. (2020), who provide a detailed study of different transfer learning and regularization approaches for low- resource machine translation. They focus on an asymmetric-resource scenario in which the target language is underresourced, but related to a higher- resource language that can be used in a multilingual setting. For example, in the English-to-Estonian task, Estonian is assumed to be a low-resource lan- guage (LRL) which is complemented by a second higher-resource target language (HRL), Finnish. Among the WMT 2020 shared tasks, the German → Upper Sorbian low-resource translation task exactly corresponds to this setup, with Czech being a high-resource language closely related to Up- per Sorbian. We adapt the approach proposed by Gr¨onroos et al. (2020) also to three slightly differ- ent scenarios: in the Upper Sorbian → German task, the low-resource language is on the source side, but can be complemented with Czech in the same way; for the English → Inuktitut task, no related high-resource language is available; and for Inuktitut → English, the low-resource language is on the source side and no high-resource language is available. Gr¨onroos et al. (2020) recommend the combina- tion of the following techniques to reach optimal translation performance in their examined setup: Scheduled multi-task learning The learning pro- cess is split in two phases. The first phase only sees data from the source and the HRL, whereas LRL data is only added in the second phase. Backtranslation The addition of synthetic data has become a staple of neural machine transla- tion. They recommend marking synthetic data and controlling its weight in the task sched- uler. Subword regularization Following Kudo (2018), each time a word is used during training, a new segmentation into subwords is sampled from the probabilistic segmentation model. Monolingual tasks In order to benefit from more easily available monolingual data and to make the model more robust to noise, they propose to include denoising sequence autoencoder tasks. A first variant applies small changes to the input side of …

Content truncated to 500 words