Findings of the WMT 2020 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT

Proceedings of the 5th Conference on Machine Translation (WMT), pages 765–771 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 765 Findings of the WMT 2020 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT Alexander Fraser Center for Information and Language Processing, LMU Munich, Germany fraser@cis.lmu.de Abstract We describe the WMT 2020 Shared Tasks in Unsupervised MT and Very Low Resource Su- pervised MT. In both tasks, the community studied German↔Upper Sorbian MT, which is a very realistic machine translation scenario (unlike the simulated scenarios used in partic- ular in much of the unsupervised MT work in the past). We were able to obtain most of the digital data available for Upper Sorbian, a mi- nority language of Germany, which was the original motivation for the Unsupervised MT shared task. As we were defining the task, we also obtained a small amount of parallel data (about 60000 parallel sentences), allowing us to offer a Very Low Supervised MT task as well. Six primary systems participated in the unsupervised shared task, two of these systems used additional data beyond the data released by the organizers. Ten primary systems partic- ipated in the very low resource supervised task. The paper discusses the background, presents the tasks and results, and discusses best prac- tices for the future. 1 Introduction There is no machine translation available for most of the approximately 7000 languages spoken on the planet Earth. This is because very limited or no parallel corpora are available. Research on unsu- pervised and very low resource machine translation is important for alleviating this problem. Unsu- pervised machine translation requires only mono- lingual data, while very low resource supervised machine translation uses very limited parallel data. At WMT 2018 and WMT 2019, the first shared task (Bojar et al., 2018) and second shared task (Barrault et al., 2019) on Unsupervised Machine Translation (UMT), were held as part of the news translation track. In both 2018 and 2019 the scenar- ios simulated low resource setups using medium or high resource languages. In 2018, the language pairs were Turkish-English, Estonian-English and German-English. In 2019, we tested unsupervised systems for German to Czech unsupervised trans- lation (where no German/Czech parallel data was allowed). In the 2020 shared task we proposed a third edi- tion on UMT, which aimed at a more realistic sce- nario, German to Upper Sorbian (and Upper Sor- bian to German) translation. Upper Sorbian is a minority language of Germany that is in the Slavic language family (e.g., related to Lower Sorbian, Czech and Polish), and we provide here most of the digital data that is available for Upper Sorbian, as far as we know. The amount of monolingual data available for Upper Sorbian is quite small (see below), making unsupervised machine translation very challenging. While working on the UMT task we were able to obtain a very small amount of parallel data (about 60000 parallel sentences) for this language pair, which allowed us to additionally offer the very low resource …

Content truncated to 500 words