Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation

Proceedings of the Third Conference on Machine Translation (WMT), Volume 1: Research Papers, pages 232–243 Belgium, Brussels, October 31 - Novermber 1, 2018. c⃝2018 Association for Computational Linguistics https://doi.org/10.18653/v1/W18-64024 Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation Zhong Zhou Carnegie Mellon University zhongzhou@cmu.edu Matthias Sperber Karlsruhe Institute of Technology matthias.sperber@kit.edu Alex Waibel Carnegie Mellon University Karlsruhe Institute of Technology alex@waibel.com Abstract We work on translation from rich-resource lan- guages to low-resource languages. The main challenges we identify are the lack of low- resource language data, effective methods for cross-lingual transfer, and the variable-binding problem that is common in neural systems. We build a translation system that addresses these challenges using eight European lan- guage families as our test ground. Firstly, we add the source and the target family labels and study intra-family and inter-family influ- ences for effective cross-lingual transfer. We achieve an improvement of +9.9 in BLEU score for English-Swedish translation using eight families compared to the single-family multi-source multi-target baseline. Moreover, we find that training on two neighboring fam- ilies closest to the low-resource language is often enough. Secondly, we construct an ab- lation study and find that reasonably good results can be achieved even with consid- erably less target data. Thirdly, we address the variable-binding problem by building an order-preserving named entity translation model. We obtain 60.6% accuracy in qualita- tive evaluation where our translations are akin to human translations in a preliminary study. 1 Introduction We work on translation from a rich-resource lan- guage to a low-resource language. There is usu- ally little low-resource language data, much less parallel data available (Duong et al., 2016; Anas- tasopoulos et al., 2017); Despite of the challenges of little data and few human experts, it has many useful applications. Applications include translat- ing water, sanitation and hygiene (WASH) guide- lines to protect Indian tribal children against water- borne diseases, introducing earthquake prepared- ness techniques to Indonesian tribal groups liv- ing near volcanoes and delivering information to the disabled or the elderly in low-resource lan- guage communities (Reddy et al., 2017; Barrett, 2005; Anastasiou and Sch¨aler, 2010; Perry and Bird, 2017). These are useful examples of trans- lating a closed text known in advance to the low- resource language. There are three main challenges. Firstly, most of previous works research on individual languages instead of collective families. Cross-lingual im- pacts and similarities are very helpful when there is little data in low-resource language (Shoemark et al., 2016; Sapir, 1921; Odlin, 1989; Cenoz, 2001; Toral and Way, 2018; De Raad et al., 1997; Hermans, 2003; Specia et al., 2016). Secondly, most of the multilingual Neural Machine Trans- lation (NMT) works assume the same amount of training data for all languages. In the low-resource case, it is important to exploit low or partial data in low-resource language to produce high quality translation. The third issue is the variable-binding problem that is common in neural systems, where “John calls Mary” is treated the same way as “Mary calls John” (Fodor and Pylyshyn, …

Content truncated to 500 words