Generalized Data Augmentation for Low-Resource Translation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5786–5796 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5786 Generalized Data Augmentation for Low-Resource Translation Mengzhou Xia, Xiang Kong, Antonios Anastasopoulos, Graham Neubig Language Technologies Institute, Carnegie Mellon University {mengzhox, xiangk, aanastas, gneubig}@andrew.cmu.edu Abstract Translation to or from low-resource languages (LRLs) poses challenges for machine transla- tion in terms of both adequacy and fluency. Data augmentation utilizing large amounts of monolingual data is regarded as an effective way to alleviate these problems. In this pa- per, we propose a general framework for data augmentation in low-resource machine transla- tion that not only uses target-side monolingual data, but also pivots through a related high- resource language (HRL). Specifically, we ex- periment with a two-step pivoting method to convert high-resource data to the LRL, mak- ing use of available resources to better approx- imate the true data distribution of the LRL. First, we inject LRL words into HRL sentences through an induced bilingual dictionary. Sec- ond, we further edit these modified sentences using a modified unsupervised machine trans- lation framework. Extensive experiments on four low-resource datasets show that under ex- treme low-resource settings, our data augmen- tation techniques improve translation quality by up to 1.5 to 8 BLEU points compared to supervised back-translation baselines.1 1 Introduction The task of Machine Translation (MT) for low re- source languages (LRLs) is notoriously hard due to the lack of the large parallel corpora needed to achieve adequate performance with current Neu- ral Machine Translation (NMT) systems (Koehn and Knowles, 2017). A standard practice to im- prove training of models for an LRL of interest (e.g. Azerbaijani) is utilizing data from a related high-resource language (HRL, e.g. Turkish). Both transferring from HRL to LRL (Zoph et al., 2016; Nguyen and Chiang, 2017; Gu et al., 2018) and 1Code is available at https://github.com/ xiamengzhou/DataAugForLRL : Available Resource : Generated Resource LRL ENG [c] HRL ENG [b] ENG [a] HRL LRL ENG LRL ENG LRL ENG [1] ENG→LRL [2] ENG→HRL [4] HRL→LRL [3] HRL→LRL Figure 1: With a low-resource language (LRL) and a related high-resource language (HRL), typical data aug- mentation scenarios use any available parallel data [b] and [c] to back-translate English monolingual data [a] and generate parallel resources ([1] and [2]). We addi- tionally propose scenarios [3] and [4], where we pivot through HRL in order to generate a LRL–ENG resource. joint training on HRL and LRL parallel data (John- son et al., 2017; Neubig and Hu, 2018) have shown to be effective techniques for low-resource NMT. Incorporating data from other languages can be viewed as one form data augmentation, and particu- larly large improvements can be expected when the HRL shares vocabulary or is syntactically similar with the LRL (Lin et al., 2019). Simple joint train- ing is still not ideal, though, considering that there will still be many words and possibly even syntactic structures that will not be shared between the most highly related languages. …

Content truncated to 500 words