MUNI-NLP Systems for Lower Sorbian-German and Lower Sorbian-Upper Sorbian Machine Translation @ WMT22

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 1111 - 1116 December 7-8, 2022 ©2022 Association for Computational Linguistics MUNI-NLP Systems for Lower Sorbian-German and Lower Sorbian-Upper Sorbian Machine Translation @ WMT22 Edoardo Signoroni and Pavel Rychlý Faculty of Informatics Masaryk University e.signoroni@mail.muni.cz, pary@fi.muni.cz Abstract We describe our neural machine translation sys- tems for the WMT22 shared task on unsuper- vised MT and very low resource supervised MT. We submit supervised NMT systems for Lower Sorbian-German and Lower Sorbian- Upper Sorbian translation in both directions. By using a novel tokenization algorithm, data augmentation techniques, such as Data Diversi- fication (DD), and parameter optimization we improve on our baselines by 10.5-10.77 BLEU for Lower Sorbian-German and by 1.52-1.88 BLEU for Lower Sorbian-Upper Sorbian. Introduction This paper describes our Machine Translation (MT) systems for the WMT22 shared task on "Unsu- pervised MT and Very Low Resource Supervised MT"1, which features translation between Lower Sorbian, Upper Sorbian, and German. Lower (dsb and Upper Sorbian (hsb) are Slavic minority lan- guages spoken in the Eastern part of Germany with 7.000 and 30.000 native speakers respectively. Text data for these languages collected and made avail- able by the Sorbian Institute and the Witaj Lan- guage Centre (Libovický and Fraser, 2021). We submit systems for Lower Sorbian-German and Lower Sorbian-Upper Sorbian in both transla- tion directions. We focused on the supervised ap- proach, using only the parallel data made available by the task organizers for all the above languages. We were able to improve on our baselines by: i. employing a new tokenization algorithm, High Fre- quency Tokenizer (HFT) (Signoroni and Rychlý, 2022); ii. augmenting the original parallel data with the Data Diversification (DD) technique by (Nguyen et al., 2020); iii. tuning the architec- ture and the parameters of the models, such as encoder/decoder depth, number of attention heads, dropout, batch size, etc. 1https://statmt.org/wmt22/unsup_and_very_low_res.html We employed HFT since it aims to obtain more meaningful subword dictionaries, while DD was chosen because it does not involve additional data apart from the original parallel corpus. Both this techniques are relevant when working with a lim- ited amount of data. This paper is structured as follows: Section 1 summarizes the data used in training; Section 2 outlines our methodology, introducing our novel tokenizer and the models we used; Section 3 sums up our final systems, while Section 4 relates and discusses the results of our experiments; Section 5 contains some final remarks. 1 Data We experiment with Lower Sorbian-German and Lower Sorbian-Upper Sorbian translation, using only the parallel data provided for each pair. The parallel data for the dsb-de and the dsb-hsb pairs consist of ∼40k and ∼62k sentences respec- tively. We use only these data, as the approach we decided to follow does not need additional data. Af- ter applying this method, our final corpus size for training is ∼360k sentences for dsb-de, and ∼560k for dsb-hsb. 2 Methodology In this section, we first present briefly our novel to- kenizer, High Frequency Tokenizer, or HFT. Then, …

Content truncated to 500 words