The AIC System for the WMT 2022 Unsupervised MT and Very Low Resource Supervised MT Task

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 1117 - 1121 December 7-8, 2022 ©2022 Association for Computational Linguistics The AIC System for the WMT 2022 Unsupervised MT and Very Low Resource Supervised MT Task Ahmad Shapiro, Mahmoud Tarek, Omar Khaled Mohamed Fayed, Ayman Khalafallah, Noha Adly Applied Innovation Center {ahmad.shapiro, mahmoudtarek, omar.khaled, m.essam, a.khalafallah, nadly}@aic.gov.eg Abstract This paper presents our submissions to WMT 22 shared task in the Unsupervised and Very Low Resource Supervised Machine Translation tasks. The task revolves around translating be- tween German ↔ Upper Sorbian (de ↔ hsb), German ↔ Lower Sorbian (de ↔ dsb) and Up- per Sorbian ↔ Lower Sorbian (hsb ↔ dsb) in a both unsupervised and supervised manner. For the unsupervised system, we trained an unsupervised phrase-based statistical machine translation (UPBSMT) system on each pair in- dependently. We pretrained a German-Slavic mBART model on the following languages Pol- ish (pl), Czech (cs), German (de), Upper Sor- bian (hsb), and Lower Sorbian (dsb). We then fine-tuned our mBART on the synthetic par- allel data generated by the (UPBSMT) model along with authentic parallel data (de ↔ pl, de ↔ cs). We further fine-tuned our unsupervised system on authentic parallel data (hsb ↔ dsb, de ↔ dsb, de ↔ hsb) to submit our supervised low-resource system. 1 Introduction Like most machine learning approaches, data can be considered the most important component of the recipe for modeling a solution for a given problem. Neural machine translation relies heavily on a large amount of training data to correctly model two lan- guages and learn the mapping between them to produce semantically and syntactically right trans- lations. However, machine translation is not avail- able for the majority of the 7000 languages spoken on the earth. This is due to the fact that parallel cor- pora are scarce or non-existent. There have been several proposals to alleviate the issue of small amounts of parallel data such as pivot translation, multilingual training, and semi-supervised training which resulted in an acceptable performance. Unsupervised machine translation became the go- to solution when lacking parallel data. The WMT 2022 Unsupervised MT Task focuses on two very low-resource languages: Upper Sorbian (HSB) and Lower Sorbian (DSB). Upper and Lower Sorbian are minority languages spoken in the federal states of Saxony and Brandenburg in Eastern Germany. With just 30,000 and 7,000 native speakers, work- ing on these languages is an extreme low-resource task, with little prospect of ever approaching the number of resources available for languages with millions of speakers. However, because they are western Slavic languages, the Sorbian languages can benefit from Czech and Polish data (Libovický and Fraser, 2021). In this paper, we describe our systems for translat- ing between German ↔ Upper Sorbian (de ↔ hsb), German ↔ Lower Sorbian (de ↔ dsb), and Upper Sorbian ↔ Lower Sorbian (hsb ↔ dsb) in a both unsupervised and supervised manner. We approach the task by combining two novel approaches for unsupervised machine translation. Influenced by (Artetxe et al., 2019), we …

Content truncated to 500 words