English-Basque Statistical and Neural Machine Translation

English-Basque Statistical and Neural Machine Translation Inigo Jauregi Unanue1,2, Lierni Garmendia Arratibel3, Ehsan Zare Borzeshi2, Massimo Piccardi1 1University of Technology Sydney (UTS), Sydney, Australia 2Capital Markets Cooperative Research Center (CMCRC), Sydney, Australia 3Professional Translator, Barcelona, Spain {ijauregi,ezborzeshi}@cmcrc.com, liernigarmendia@gmail.com, massimo.piccardi@uts.edu.au Abstract Neural Machine Translation (NMT) has attracted increasing attention in the recent years. However, it tends to require very large training corpora which could prove problematic for languages with low resources. For this reason, Statistical Machine Translation (SMT) continues to be a popular approach for low-resource language pairs. In this work, we address English-Basque translation and compare the performance of three contemporary statistical and neural machine translation systems: OpenNMT, Moses SMT and Google Translate. For evaluation, we employ an open-domain and an IT-domain corpora from the WMT16 resources for machine translation. In addition, we release a small dataset (Berriak) of 500 highly-accurate English-Basque translations of complex sentences useful for a thorough testing of the translation systems. Keywords: Neural Machine Translation, Statistical Machine Translation, English-Basque, Basque 1. Introduction The advent of deep neural networks in natural language processing (NLP) has led to significant progress in a vari- ety of classification tasks, including named-entity recogni- tion (Lample et al., 2016), answer sentence selection (Lowe et al., 2017) and natural language inference (Wang et al., 2017). Deep neural networks have also started to obtain promising results in NLP, mainly in Neural Machine Trans- lation (NMT) (Sutskever et al., 2014; Bahdanau et al., 2014; Vaswani et al., 2017; Wu et al., 2016). In NMT, the model receives a sentence in a source language as in- put and generates its translation word by word in a tar- get language. NMT has outperformed previous transla- tion systems in many language pairs (e.g., German-English, French-English). However, in order to reach high accuracies, neural trans- lation systems tend to require very large parallel training corpora (Koehn and Knowles, 2017). As a matter of fact, such corpora are not yet available for many language pairs. When the training data are relatively small, other, more tra- ditional approaches such as Statistical Machine Translation (SMT) (Koehn et al., 2007) seem to be more accurate. Sev- eral ideas have been proposed in order to mollify this is- sue including multi-lingual systems with zero-shot transla- tion (Johnson et al., 2016), transfer learning (Zoph et al., 2016) and back-translations (Sennrich et al., 2016). How- ever, their general effectiveness still requires wider evalua- tion. For this work, we have selected a low-resource language pair, English-Basque (abbreviation: en-eu), and used it as a case study for statistical and neural machine trans- lation. Past machine translators for the Basque language have mainly used rule-based (Mayor et al., 2011) and sta- tistical approaches (de Ilarraza et al., 2008; Del Gaudio et al., 2016; Stroppa et al., 2006). In our work, we compare three different systems: OpenNMT, an open-source NMT system; Moses SMT, an open-source SMT system; and Google Translate, a publicly-available commercial system which uses either SMT or NMT models depending on the language pair. The first two have …

Content truncated to 500 words