Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 802–812 August 1–6, 2021. ©2021 Association for Computational Linguistics 802 Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data Wei-Jen Ko1∗, Ahmed El-Kishky2∗, Adithya Renduchintala3, Vishrav Chaudhary3, Naman Goyal3, Francisco Guzm´an3, Pascale Fung4, Philipp Koehn5, Mona Diab3 1University of Texas at Austin, 2Twitter Cortex, 3Facebook AI 4The Hong Kong University of Science and Technology, 5Johns Hopkins University wjko@utexas.edu, aelkishky@twitter.com {adirendu,vishrav,naman,fguzman,mdiab}@fb.com pascale@ece.ust.hk, phi@jhu.edu Abstract The scarcity of parallel data is a major obsta- cle for training high-quality machine transla- tion systems for low-resource languages. For- tunately, some low-resource languages are lin- guistically related or similar to high-resource languages; these related languages may share many lexical or syntactic structures. In this work, we exploit this linguistic overlap to facilitate translating to and from a low- resource language with only monolingual data, in addition to any parallel data in the re- lated high-resource language. Our method, NMT-Adapt, combines denoising autoencod- ing, back-translation and adversarial objec- tives to utilize monolingual data for low- resource adaptation. We experiment on 7 lan- guages from three different language families and show that our technique significantly im- proves translation into low-resource language compared to other translation baselines. 1 Introduction While machine translation (MT) has made incredi- ble strides due to the advent of deep neural machine translation (NMT) (Sutskever et al., 2014; Bah- danau et al., 2014) models, this improvement has been shown to be primarily in well-resourced lan- guages with large available parallel training data. However with the growth of internet commu- nication and the rise of social media, individuals worldwide have begun communicating and produc- ing content in their native low-resource languages. Many of these low-resource languages are closely related to a high-resource language. One such ex- ample are “dialects”: variants of a language tradi- tionally considered oral rather than written. Ma- chine translating dialects using models trained on ∗This work was conducted while author was working at Facebook AI the formal variant of a language (typically the high- resource variant which is sometimes considered the “standardized form”) can pose a challenge due to the prevalence of non standardized spelling as well significant slang vocabulary in the dialectal variant. Similar issues arise from translating a low-resource language using a related high-resource model (e.g., translating Catalan with a Spanish MT model). While an intuitive approach to better translating low-resource related languages could be to obtain high-quality parallel data. This approach is often infeasible due to lack specialized expertise or bilin- gual translators. The problems are exacerbated by issues that arise in quality control for low-resource languages (Guzm´an et al., 2019). This scarcity motivates our task of learning machine translation models for low-resource languages while leverag- ing readily available data such as parallel data from a closely related language or monolingual data in the low-resource language.1 The use of monolingual data when little to no parallel data is available has been investigated …

Content truncated to 500 words