Investigating Phrase-Based and Neural-Based Machine Translation on Low-Resource Settings

31st Pacific Asia Conference on Language, Information and Computation (PACLIC 31), pages 384–391 Cebu City, Philippines, November 16-18, 2017 Copyright c⃝2017 Hai Long Trieu, Vu Tran and Nguyen Le Minh Investigating Phrase-Based and Neural-Based Machine Translation on Low-Resource Settings Hai-Long Trieu, Duc-Vu Tran, Le-Minh Nguyen Japan Advanced Institute of Science and Technology {trieulh,vu.tran,nguyenml}@jaist.ac.jp Abstract Neural-based and phrase-based methods have shown the effectiveness and promising results in the development of current machine transla- tion. The two methods are compared on some European languages, which show the advan- tages of the neural machine translation. Nev- ertheless, there are few work of comparing the two methods on low-resource languages, which there are only small bilingual corpora. The problem of unavailable large bilingual corpora causes a bottleneck for machine trans- lation for such language pairs. In this paper, we present a comparison of the phrase-based and neural-based machine translation methods on several Asian language pairs: Japanese- English, Indonesian-Vietnamese, and English- Vietnamese. Additionally, we extracted a bilingual corpus from Wikipedia to enhance machine translation performance. Experimen- tal results showed that when using the ex- tracted corpus to enlarge the training data, neural machine translation models achieved the higher improvement and outperformed the phrase-based models. This work can be useful as a basis for further development of machine translation on the low-resource languages. 1 Introduction Recent approaches have shown the promising results in the development of machine translation. Dur- ing a long period from statistical models (Brown et al., 1990; Brown et al., 1993) to phrase-based mod- els (Och et al., 1999; Koehn et al., 2003; Chiang, 2005) to recent neural-based methods (Sutskever et al., 2014; Cho et al., 2014), the phrase-based and neural-based become dominant methods in current machine translation. Statistical machine translation (SMT) systems achieve a high performance in many typologically diverse language pairs (Bojar et al., 2013). SMT can be applied to any pair of languages with minimal engineering effort (Bisazza and Fed- erico, 2016). Meanwhile, neural machine transla- tion (NMT) has obtained the state-of-the-art perfor- mance in machine translation for several languages including Czech-English, German-English, English- Romanian (Sennrich et al., 2016a). NMT has been proposed recently as a promising framework for machine translation, which learns sequence- to-sequence mapping based on two recurrent neu- ral networks (Sutskever et al., 2014; Cho et al., 2014), called encoder-decoder networks. In a ba- sic encoder-decoder network, the dimension of the context vector in the encoder is fixed, which leads to a low performance when translating for long sen- tences. In order to overcome the problem, (Bah- danau et al., 2015) proposed a method called at- tention mechanism, in which the model encodes the most relevant information in an input sentence rather than a whole input sentence into the fixed length context vector. NMT models with the at- tention mechanism have achieved significantly im- provement in many language pairs (Jean et al., 2015; Gulcehre et al., 2015; Luong et al., 2015). SMT and NMT models have shown success- fully in language pairs in which large bilingual corpora …

Content truncated to 500 words