The LMU Munich System for the WMT20 Very Low Resource Supervised MT Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 1104–1111 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 1104 The LMU Munich System for the WMT20 Very Low Resource Supervised MT Task Jindˇrich Libovick´y and Viktor Hangya and Helmut Schmid and Alexander Fraser Center for Information and Language Processing LMU Munich {libovicky,hangya,schmid,fraser}@cis.lmu.de Abstract We present our systems for the WMT20 Very Low Resource MT Task for translation be- tween German and Upper Sorbian. For train- ing our systems, we generate synthetic data by both back- and forward-translation. Ad- ditionally, we enrich the training data with German-Czech translated from Czech to Up- per Sorbian by an unsupervised statistical MT system incorporating orthographically similar word pairs and transliterations of OOV words. Our best translation system between German and Sorbian is based on transfer learning from a Czech-German system and scores 12 to 13 BLEU higher than a baseline system built us- ing the available parallel data only. 1 Introduction In this paper, we describe systems for translation between German and Upper Sorbian developed at LMU Munich for the WMT20 shared task on very low-resource supervised MT. Upper Sorbian is a minority language spoken by around 30,000 people in today’s German state of Saxony. With such a small number of speak- ers, machine translation and automatic processing of Sorbian is an inherently low-resource problem without any chance that the resources available for Sorbian would ever approach the size of resources for languages spoken by millions of people. On the other hand, being a Western Slavic language related to Czech and Polish, it is possible to take advantage of relatively rich resources collected for these two languages. The German-Sorbian systems presented in this paper are neural machine translation (NMT) systems based on the Transformer architecture (Vaswani et al., 2017). We experiment with vari- ous data preparation and augmentation techniques: back-translation (Sennrich et al., 2016b), finetun- ing systems trained for translation between Czech and German (Kocmi and Bojar, 2018), and data augmentation by including German-Czech paral- lel data with the Czech side translated to Upper Sorbian by an unsupervised system that includes an unsupervised transliteration model for guessing how to translate out-of-vocabulary Czech words to Upper Sorbian. Our experiments show the importance of data augmentation via stochastic pre-processing and synthetic data generation. The best systems were trained by transfer-learning from a Czech-German system. However, compared to data augmentation, transfer learning from Czech-German translation only produces a minor improvement. Based on the preliminary shared task results, the presented sys- tems scored on the 4th place among 10 competing teams in the shared task. 2 Related Work Until recently, phrase-based approaches were be- lieved to be more suitable for low-resource trans- lation. Koehn and Knowles (2017) claimed that a parallel dataset of at least 107 tokens is required for NMT to outperform phrase-based MT. This view was also supported by the results of Artetxe et al. (2018b) and Lample et al. (2018), who showed that phrase-based approaches work well for unsuper- vised MT, at least …

Content truncated to 500 words