Iterative Data Augmentation for Neural Machine Translation: a Low Resource Case Study for English-Telugu

Iterative Data Augmentation for Neural Machine Translation: a Low Resource Case Study for English–Telugu Sandipan Dandapat and Christian Federmann Microsoft AI & Research {sadandap,chrife}@microsoft.com Abstract Telugu is the fifteenth most commonly spoken language in the world with an estimated reach of 75 million people in the Indian subcontinent. At the same time, it is a severely low resourced language. In this paper, we present work on English–Telugu general domain machine translation (MT) systems using small amounts of parallel data. The baseline statistical (SMT) and neural MT (NMT) systems do not yield acceptable translation quality, mostly due to limited resources. However, the use of synthetic parallel data (generated using back translation, based on an NMT engine) significantly improves translation quality and allows NMT to outperform SMT. We extend back translation and propose a new, iterative data augmentation (IDA) method. Filtering of synthetic data and IDA both further boost translation quality of our final NMT systems, as measured by BLEU scores on all test sets and based on state- of-the-art human evaluation. 1 Introduction In the past two decades, machine translation (MT) has shown very promising results, most of which have been achieved using data-driven techniques. In recent years, the data-driven paradigm of MT is largely dominated by neural machine trans- lation (NMT) and showing significant success over its predecessor statistical machine translation (SMT) (Bahdanau et al., 2014; Bojar et al., 2017). c⃝ 2018 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. The performance of any data-driven approach to MT mostly depends on the amount of parallel cor- pora available to train them. This problem is ex- acerbated by NMT, which generally needs larger quantities of parallel data and is less robust to noisy data. Unfortunately, large amounts of readily available parallel resources exist only for a small number of languages, e.g., OPUS (Tiedemann and Nygaard, 2004) and Europarl (Koehn, 2005), with only very few sources of Indic language data. Indic language MT is difficult due to complex linguistic structure and lack of good quality data. Most of the Indic languages are leading languages of the world in terms of number of speakers but are very poorly resourced (i.e., only very little machine-readable parallel text exists) so build- ing a general domain data-driven MT system is a challenging problem. Also, Indic languages do not have enough comparable resources to explore extraction of useful parallel content from the same (Irvine and Callison-Burch, 2013). Lastly, due to the usage of multiple fonts and encodings, a significant portion of the web data cannot be used to extract parallel data for training. Telugu is no exception to this. Lack of large, high quality parallel resources makes the development of gen- eral purpose MT systems much harder for Telugu compared to other, resource rich languages, more specifically when building NMT-based models. One of the major problems with training an NMT system on little data, especially when train- ing an engine for general usage (i.e., not domain specific), …

Content truncated to 500 words