NRC-CNRC Systems for Upper Sorbian-German and Lower Sorbian-German Machine Translation 2021

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 999–1008 November 10–11, 2021. ©2021 Association for Computational Linguistics 999 NRC-CNRC Systems for Upper Sorbian–German and Lower Sorbian–German Machine Translation 2021 Rebecca Knowles∗˜ and Samuel Larkin∗ National Research Council Canada {Rebecca.Knowles, Samuel.Larkin}@nrc-cnrc.gc.ca Abstract We describe our neural machine translation systems for the 2021 shared task on Unsuper- vised and Very Low Resource Supervised MT, translating between Upper Sorbian and Ger- man (low-resource) and between Lower Sor- bian and German (unsupervised). The sys- tems incorporated data filtering, backtransla- tion, BPE-dropout, ensembling, and transfer learning from high(er)-resource languages. As measured by automatic metrics, our systems showed strong performance, consistently plac- ing first or tied for first across most metrics and translation directions. 1 Introduction This work describes our machine translation (MT) systems for translating between Upper Sorbian– German and Lower Sorbian–German (all transla- tion directions). We focused primarily on the su- pervised task of Upper Sorbian–German, and then applied those systems to the task of building simple Lower Sorbian–German systems.1 Upper Sorbian and Lower Sorbian are Slavic mi- nority languages spoken in eastern Germany, along- side German. The shared task data was provided to the organizers through collaborations with the Sorbian Institute2 and the Witaj Language Cen- tre,3 as described in Fraser (2020), to which we direct interested readers for additional information on the languages and data. Following the 2020 shared task, the Witaj Language Centre released a publicly-available Sorbian–German MT system sotra (Witaj Language Centre, 2021) based on Moses (Koehn et al., 2007) and OpenNMT (Klein et al., 2017).4 ∗Both authors contributed equally to this work. 1We abbreviate language names as follows: cs (Czech), de (German), dsb (Lower Sorbian), and hsb (Upper Sorbian). 2https://www.serbski-institut.de/en/ Institute/ 3https://www.witaj-sprachzentrum.de/ 4https://sotra.app We provide an overview of the data, preprocess- ing, and model architectures in Sections 2, 3, and 4. We then discuss baselines, systems, experiments in monolingual filtering, and backtranslation (all focused on Upper Sorbian–German) in Sections 5, 6, 7, and 8. In Section 9, we discuss how we ap- plied and finetuned our existing Upper Sorbian MT systems for the task of translating Lower Sorbian. Section 10 discusses additional experiments with negative results. Finally, Sections 11 and 12 sum- marize the final systems and our conclusions. 2 Data We used all provided parallel German–Upper Sor- bian data and all monolingual Upper Sorbian data (after filtering), along with German–Czech parallel data from Open Subtitles (Lison and Tiedemann, 2016),5 DGT (Tiedemann, 2012; Steinberger et al., 2012), JW300 (Agi´c and Vuli´c, 2019), Europarl v10 (Koehn, 2005), News-Commentary v15, and WMT-News.6 We also used the monolingual Upper Sorbian Web, Witaj and Sorbian Institute datasets as well as the Lower Sorbian monolingual data (the latter for Lower Sorbian tasks only).7 We used the provided devel sets for development, and the devel_test systems for measuring progress and choosing which systems to submit. 3 Preprocessing and Postprocessing As preprocessing, we first clean all of the available training data (but not development or test data) using clean-utf8-text.pl with the -no-phrase-sep flag from …

Content truncated to 500 words