The LMU Munich Systems for the WMT21 Unsupervised and Very Low-Resource Translation Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 989–994 November 10–11, 2021. ©2021 Association for Computational Linguistics 989 The LMU Munich Systems for the WMT21 Unsupervised and Very Low-Resource Translation Task Jindˇrich Libovický and Alexander Fraser Center for Information and Language Processing LMU Munich {libovicky,fraser}@cis.lmu.de Abstract We present our submissions to the WMT21 shared task in Unsupervised and Very Low- Resource machine translation between Ger- man and Upper Sorbian, German and Lower Sorbian, and Russian and Chuvash. Our low-resource systems (German↔Upper Sor- bian, Russian↔Chuvash) are pre-trained on high-resource pairs of related languages. We fine-tune those systems using the available authentic parallel data and improve by it- erated back-translation. The unsupervised German↔Lower Sorbian system is initial- ized by the best Upper Sorbian system and improved by iterated back-translation using monolingual data only. 1 Introduction In this paper, we describe systems for translation between German (de) and Upper Sorbian (hsb), German (de) and Lower Sorbian (dsb), and Russian (ru) and Chuvash (cv) developed at LMU Munich for the WMT21 shared task on unsupervised and very low resource machine translation (MT). Upper Sorbian is a minority language spoken by around 30,000 people in today’s German fed- eral state of Saxony, Lower Sorbian has around 7,000 speakers and is spoken in the German federal state of Brandenburg. With such a small number of speakers, machine translation and automatic pro- cessing of Sorbian language is an inherently low- resource problem without any chance that the re- sources available for Sorbian would ever approach the size of resources for languages spoken by mil- lions of people. On the other hand, being Western Slavic languages related to Czech and Polish, it is possible to take advantage of relatively rich re- sources collected for these two languages. Unlike our last year’s submission for Upper Sor- bian (Libovický et al., 2020), we decided not to use synthetic data from unsupervised translation between Czech and Upper Sorbian and only did iterative back-translation. Despite having more authentic parallel data than last year, our system reaches approximately the same translation quality. Our Upper Sorbian systems ranked third out of six systems in the official ranking. We leverage the relatedness between the Sorbian languages and use the Upper Sorbian system as a starting point for iterative back-translation using monolingual data only. Our Lower Sorbian Sys- tems ranked second (de→dsb) and third (dbs→de) out of four teams in the official ranking. Chuvash is a minority language spoken in the Volga region in the southwest of Russia. Although it uses the Cyrillic script, it is not related to eastern Slavic languages, but it is a Turkic language, rel- atively isolated in the Turkic language family. As a language with the highest number of speakers in this shared task, it also has the highest amount of available parallel data. We adopt a similar approach as for German-Upper Sorbian translation and pre- train our models on the related Kazakh language. In addition, we experiment with character-level models in the hope that they will be particularly …

Content truncated to 500 words