Machine Translation of Low-Resource Indo-European Languages

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 347–353 November 10–11, 2021. ©2021 Association for Computational Linguistics 347 Machine Translation of Low-Resource Indo-European Languages Wei-Rui Chen Muhammad Abdul-Mageed Natural Language Processing Lab The University of British Columbia {weirui.chen,muhammad.mageed}@ubc.ca Abstract In this work, we investigate methods for the challenging task of translating between low- resource language pairs that exhibit some level of similarity. In particular, we consider the utility of transfer learning for translating between several Indo-European low-resource languages from the Germanic and Romance language families. In particular, we build two main classes of transfer-based systems to study how relatedness can benefit the translation per- formance. The primary system fine-tunes a model pre-trained on a related language pair and the contrastive system fine-tunes one pre- trained on an unrelated language pair. Our experiments show that although relatedness is not necessary for transfer learning to work, it does benefit model performance. 1 Introduction Machine translation (MT) is currently one of the hot application areas of deep learning, with neu- ral machine translation (NMT) achieving outstand- ing performance where large amounts of paral- lel data are available (Koehn and Knowles, 2017; Ranathunga et al., 2021; Luong et al., 2015a; Nguyen and Chiang, 2017). In low-resource set- tings, transfer learning methods have proven useful for improving system performance (Zoph et al., 2016; Nguyen and Chiang, 2017; Kocmi and Bo- jar, 2018). In this work, we focus on studying NMT, including in the low-resource scenario. In particular, we focus our attention on investigating the effect of language relatedness on the transfer process. We define relatedness of a pair of lan- guages based on belonging to the same language family. That is, by ‘related’ we mean ‘within the same language family’ whereas by ‘unrelated’ we mean ‘belong to two different language families’. For example, we call English and Swedish related since they belong to the Germanic language family but English and French not related since the latter belongs to the Romance language family. As an analogy to human learning, we would like to ask: if there are two translators (pre-trained models), for example one Catalan → Spanish translator and one Catalan → English translator, will they (af- ter extra training, i.e., fine-tuning/transfer learning process) have different abilities to translate from Catalan into Occitan? If the Catalan-Spanish trans- lator proves to perform better Catalan → Occitan, we may attribute this to Spanish and Occitan being members of the Romance language family while English being a member of the, Germanic, different family. Of particular interest to us are two sets of lan- guages belonging to two different language fami- lies, one set to Romance and the other set to Ger- manic. For the former set, we take Catalan (ca), Italian (it), Occitan (oc), Romanian (ro), and Span- ish (es); and we take English (en) for the latter set. We note that both Romance and Germanic are two branches of the larger Indo-European lan- guage family, and hence there are some level of relatedness between all the …

Content truncated to 500 words