Findings of the WMT 2021 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 726–732 November 10–11, 2021. ©2021 Association for Computational Linguistics 726 Findings of the WMT 2021 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT Jindˇrich Libovický and Alexander Fraser Center for Information and Language Processing LMU Munich {libovicky,fraser}@cis.lmu.de Abstract We present the findings of the WMT2021 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT. Within the task, the community studied very low resource translation between German and Upper Sor- bian, unsupervised translation between Ger- man and Lower Sorbian and low resource translation between Russian and Chuvash, all minority languages with active language com- munities working on preserving the languages, who are partners in the evaluation. Thanks to this, we were able to obtain most digital data available for these languages and offer them to the task participants. In total, six teams partic- ipated in the shared task. The paper discusses the background, presents the tasks and results, and discusses best practices for the future. 1 Introduction For some languages, machine translation (MT) reached such a high quality that allows a discussion of whether and under what circumstance human parity might have been reached (Popel et al., 2020; Läubli et al., 2020). This is the case, however, for only a small minority of the world’s language. For most of the 7k languages spoken in the world only very limited resources exist. The goal of the WMT Shared Task on Unsupervised and Very Low Re- source MT is to promote research on methods for MT that alleviate such data sparsity in a real-world setup. A task on unsupervised MT was already held at WMT in 2018 (Bojar et al., 2018) and 2019 (Bar- rault et al., 2019), where the lack of parallel data was simulated on high-resource language pairs: English–German in 2018 and German–Czech in 2019. Starting from last year, we cooperate with lo- cal communities working on preserving their lan- guages. In cooperation with the Sorbian Insti- tute1 and the Witaj Sprachzentrum2, we offered a shared task in translation between German and Upper Sorbian in low-resource and unsupervised tracks (Fraser, 2020). For this year, we kept the low-resource track for Upper Sorbian and added un- supervised translation between German and Lower Sorbian. Upper and Lower Sorbian are minority languages spoken in the east part of Germany in the federal states of Saxony and Brandenburg. Hav- ing only 30k and 7k native speakers, processing of the languages is an inherently low-resource prob- lem, without any chance that the size of available resources would ever get close to the size of re- sources available for languages with millions of speakers. On the other hand, being western Slavic languages, the Sorbian languages can take advan- tage of existing resources for Czech and Polish. Additionally, in cooperation with the Chuvash Language Laboratory3, we added another low- resource task, translation between Russian and Chuvash. Chuvash is a minority Turkic language spoken by approximately one million people in the Volga region in …

Content truncated to 500 words