CUNI systems for WMT21: Multilingual Low-Resource Translation for Indo-European Languages Shared Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 354–361 November 10–11, 2021. ©2021 Association for Computational Linguistics 354 CUNI systems for WMT21: Multilingual Low-Resource Translation for Indo-European Languages Shared Task Josef Jon and Michal Novák and João Paulo Aires and Dušan Variš and Ondˇrej Bojar Charles University {jon,mnovak,aires,varis,bojar}@ufal.mff.cuni.cz Abstract This paper describes Charles University sub- mission for Multilingual Low-Resource Trans- lation for Indo-European Languages shared task at WMT21. We competed in translation from Catalan into Romanian, Italian and Occi- tan. Our systems are based on shared multilin- gual model. We show that using joint model for multiple similar language pairs improves upon translation quality in each pair. We also demonstrate that chararacter-level bilin- gual models are competitive for very similar language pairs (Catalan-Occitan) but less so for more distant pairs. We also describe our experiments with multi-task learning, where aside from a textual translation, the models are also trained to perform grapheme-to-phoneme conversion. 1 Introduction The goal of the task was to translate text from Catalan into Occitan, Italian and Romanian. Ad- ditionally, use of parallel corpora which combine the evaluated languages with English, French, Por- tuguese and Spanish was permitted. The choice of the languages from the same family invites to ex- plore how to take advantage of similarities between the languages. One way to exploit similarities between the lan- guages translated by an NMT model is to train a single joint model for multiple languages. This way, parameters representing rules and features which are common for multiple languages can be shared and better estimated due to a larger amount of training examples related to them. Another approach which can be effective when source and target languages are very similar is character-level processing of the text. Since most of the differences between Catalan and Occitan are straightforward orthographic variations, we hypoth- esize that the translation model would benefit from being able to manipulate the text at character level instead of larger subwords. We also explore making use of language sim- ilarity in spoken form, aside from written form. Languages from the same language group may be more mutually intelligible in their spoken form rather than in the written form. For instance, based on our anecdotal observations, native speakers of Czech report better understanding of spoken rather than written Polish. This is mainly due to Polish orthography, which is regular but uses various di- graphs, making Polish texts less comprehensible for common Czech speakers. Phonemic representa- tions may be even more helpful for languages with irregular spelling. Instead of using automatically acquired phone- mic representation as one of the inputs, we rather focus on strengthening robustness of our translation models by teaching them to produce this represen- tation as an additional task. Some of our models are thus trained to provide machine translation as well as grapheme-to-phoneme conversion (G2P) of the source. 2 Main features of our approach The core of our approach lies in leveraging multi- lingual training data, various subword granularity and phonemic representation of texts …

Content truncated to 500 words