Revisiting Low-Resource Neural Machine Translation: A Case Study

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 211–221 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 211 Revisiting Low-Resource Neural Machine Translation: A Case Study Rico Sennrich1,2 Biao Zhang1 1School of Informatics, University of Edinburgh rico.sennrich@ed.ac.uk, B.Zhang@ed.ac.uk 2Institute of Computational Linguistics, University of Zurich Abstract It has been shown that the performance of neu- ral machine translation (NMT) drops starkly in low-resource conditions, underperforming phrase-based statistical machine translation (PBSMT) and requiring large amounts of aux- iliary data to achieve competitive results. In this paper, we re-assess the validity of these results, arguing that they are the result of lack of system adaptation to low-resource settings. We discuss some pitfalls to be aware of when training low-resource NMT systems, and re- cent techniques that have shown to be espe- cially helpful in low-resource settings, result- ing in a set of best practices for low-resource NMT. In our experiments on German–English with different amounts of IWSLT14 training data, we show that, without the use of any aux- iliary monolingual or multilingual data, an op- timized NMT system can outperform PBSMT with far less data than previously claimed. We also apply these techniques to a low-resource Korean–English dataset, surpassing previously reported results by 4 BLEU. 1 Introduction While neural machine translation (NMT) has achieved impressive performance in high-resource data conditions, becoming dominant in the field (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017), recent research has ar- gued that these models are highly data-inefficient, and underperform phrase-based statistical ma- chine translation (PBSMT) or unsupervised meth- ods in low-data conditions (Koehn and Knowles, 2017; Lample et al., 2018b). In this paper, we re-assess the validity of these results, arguing that they are the result of lack of system adaptation to low-resource settings. Our main contributions are as follows: • we explore best practices for low-resource 106 107 108 0 10 20 30 21.8 23.4 24.9 26.2 26.9 27.9 28.6 29.2 29.6 30.1 30.4 16.4 18.1 19.6 21.2 22.2 23.5 24.7 26.1 26.9 27.8 28.6 1.6 7.2 11.9 14.7 18.2 22.4 25.7 27.4 29.2 30.3 31.1 Corpus Size (English Words) BLEU Scores with Varying Amounts of Training Data Phrase-Based with Big LM Phrase-Based Neural Figure 3: BLEU scores for English-Spanish sys- tems trained on 0.4 million to 385.7 million words of parallel data. Quality for NMT starts much lower, outperforms SMT at about 15 mil- lion words, and even beats a SMT system with a big 2 billion word in-domain language model un- der high-resource conditions. How do the data needs of SMT and NMT com- pare? NMT promises both to generalize better (ex- ploiting word similary in embeddings) and condi- tion on larger context (entire input and all prior output words). We built English-Spanish systems on WMT data,7 about 385.7 million English words paired with Spanish. To obtain a learning curve, we used 1 1024, 1 512, ..., 1 2, and all of the data. For SMT, the language …

Content truncated to 500 words