Improving Similar Language Translation With Transfer Learning

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 273–278 November 10–11, 2021. ©2021 Association for Computational Linguistics 273 Improving Similar Language Translation With Transfer Learning Ife Adebara Muhammad Abdul-Mageed Natural Language Processing Lab The University of British Columbia {ife.adebara,muhammad.mageed}@ubc.ca Abstract We investigate transfer learning based on pre- trained neural machine translation models to translate between (low-resource) similar lan- guages. This work is part of our contribu- tion to the WMT 2021 Similar Languages Translation Shared Task where we submitted models for different language pairs, includ- ing French-Bambara, Spanish-Catalan, and Spanish-Portuguese in both directions. Our models for Catalan-Spanish (82.79 BLEU) and Portuguese-Spanish (87.11 BLEU) rank top 1 in the official shared task evaluation, and we are the only team to submit models for the French-Bambara pairs. 1 Introduction We present the findings from our participation in the WMT 2021 Similar Language Translation shared task 2021, which focused on translation between similar language pairs in low-resource set- tings. The similar languages task focuses on build- ing machine translation (MT) systems for transla- tion between pairs of similar languages, without English as a pivot language. Similarity between languages interacts with MT quality, usually positively (Adebara et al., 2020). Languages described as similar usually share cer- tain levels of mutual intelligibility. Depending on the level of closeness, certain languages may share orthography, lexical, syntactic, and/or semantic structures which may facilitate translation between pairs. However, (a) scarcity of parallel data that can be used for training MT models remains a bot- tleneck. Even high-resource pairs can suffer from (b) low-data quality. That is, available data is not always guaranteed to be actual bitext with target standing as a translation of source. In fact, some open resources such as OPUS (Tiedemann, 2012a; Tiedemann et al., 2015) can suffer from noise such as when the source and target sentences belong to the same language. In this work, we tackle both (a) scarcity and (b) low-data quality. For a, we use simple knowledge transfer from already trained MT models to the downstream pair. For b, we use a simple procedure of language identification to remove noisy bitext where both the source and tar- get are detected to be the same language or where source or target is identified as a different language from what it is expected to be. The models we develop are for Spanish to Cata- lan (ES-CA), Catalan to Spanish (CA-ES), Spanish to Portuguese (PT-ES), Portuguese to Spanish (PT- ES), French to Bambara (FR-BM), and Bambara to French (BM-FR) language pairs1. Whenever possi- ble, we choose an available MT model trained with the same source and target languages as our pair of interest. In cases where no such a model exists, we pick a model with either the source or the target language as our intended pair (Section 5). To show the utility of our transfer learning approach to the problem, we also train on one pair from scratch (which we treat as a baseline). We experiment with tokenized (primary mod- els) and …

Content truncated to 500 words