Handling Syntactic Divergence in Low-resource Machine Translation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1388–1394, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1388 Handling Syntactic Divergence in Low-resource Machine Translation Chunting Zhou, Xuezhe Ma, Junjie Hu, Graham Neubig Language Technologies Institute Carnegie Mellon University {chuntinz,xuezhem,junjieh,gneubig}@cs.cmu.edu Abstract Despite impressive empirical successes of neural machine translation (NMT) on standard benchmarks, limited parallel data impedes the application of NMT models to many language pairs. Data augmentation methods such as back-translation make it possible to use monolingual data to help alleviate these issues, but back-translation itself fails in extreme low-resource sce- narios, especially for syntactically diver- gent languages. In this paper, we propose a simple yet effective solution, whereby target-language sentences are re-ordered to match the order of the source and used as an additional source of training- time supervision. Experiments with sim- ulated low-resource Japanese-to-English, and real low-resource Uyghur-to-English scenarios find significant improvements over other semi-supervised alternatives1 1 Introduction While neural machine translation (NMT; Bah- danau et al. (2015); Vaswani et al. (2017)) now represents the state of the art in the majority of large-scale MT benchmarks (Bojar et al., 2017), it is highly dependent on the availability of copious parallel resources; NMT under-performs previ- ous phrase-based methods when the training data is small (Koehn and Knowles, 2017). Unfortu- nately, million-sentence parallel corpora are of- ten unavailable for many language pairs. Con- versely, monolingual sentences, particularly in En- glish, are often much easier to find, making semi- supervised approaches that can use monolingual data a desirable solution to this problem.2 1https://github.com/violet-zct/pytorch-.reorder-nmt 2Unsupervised MT (Artetxe et al., 2018a) has achieved success on simulated low-resource scenarios with related lan- 私 は 新しい ⾞車車 を 買った 。 I bought a new car . I var_1 a new car var_2 bought . Reference Japanese: Japanese-ordered English: English: Figure 1: An English sentence re-ordered into Japanese order using the rule-based method of Isozaki et al. (2010b), and its reference Japanese translation. Semi-supervised approaches for NMT are of- ten based on automatically creating pseudo- parallel sentences through methods such as back- translation (Irvine and Callison-Burch, 2013; Sen- nrich et al., 2016) or adding an auxiliary auto- encoding task on monolingual data (Cheng et al., 2016; He et al., 2016; Currey et al., 2017). How- ever, both methods have problems with low- resource and syntactically divergent language pairs. Back translation assumes enough data to create a functional NMT system, an unrealistic re- quirement in low-resource scenarios, while auto- encoding target sentences by definition will not be able to learn source-target word reordering. This paper proposes a method to create pseudo- parallel sentences for NMT for language pairs with divergent syntactic structures. Prior to NMT, word reordering was a major challenge for statis- tical machine translation (SMT), and many tech- niques have emerged over the years to address this challenge (Xia and McCord, 2004; Bisazza and Federico, 2016). Importantly, even simple heuristic reordering methods with a few hand- …

Content truncated to 500 words