Iterative Multilingual Neural Machine Translation for Less-Common and Zero-Resource Language Pairs

Iterative Multilingual Neural Machine Translation for Less-Common and Zero-Resource Language Pairs Minh-Thuan Nguyen, Phuong-Thai Nguyen, Van-Vinh Nguyen, Minh-Cong Nguyen Hoang Department of Computer Science, University of Engineering and Technology, VNU Hanoi {thuannm, thainp, vinhnv}@vnu.edu.vn minhcongnguyen1508@gmail.com Abstract Research on providing machine translation systems for unseen language pairs is gaining increasing attention in recent years. However, the quality of their systems is poor for most language pairs, especially for less-common pairs such as Khmer-Vietnamese. In this paper, we show a simple iterative training- generating-filtering-training process that uti- lizes all available pivot parallel data to gen- erate synthetic data for unseen directions. In addition, we propose a filtering method based on word alignments and the longest parallel phrase to filter out noise sentence pairs in the synthetic data. Experiment re- sults on zero-shot Khmer→Vietnamese and Indonesian→Vietnamese directions show that our proposed model outperforms some strong baselines and achieves a promising result under the zero-resource condition on ALT benchmarks. Besides, the results also indicate that our model can easily improve their quality with a small amount of real parallel data. 1 Introduction Neural Machine Translation (NMT) has recently achieved impressive performance on high-resource language pairs which have large amounts of parallel training data (Wu et al., 2016) (Vaswani et al., 2017). However, these systems still work poorly when the parallel data is low or unavailable. Research on zero-resource language pairs is gaining much at- tention in recent years, and it has been found to use pivot language, zero-shot NMT, or zero-resource NMT approaches to deal with the translation of un- seen language pairs. In pivot language approaches, sentences are first translated from the source language into the pivot language through a source-pivot system, and then from the pivot language into the target language by using a pivot-target system. Although this simple process has shown strong translation performance (Johnson et al., 2017), it has a few limitations. The pivoting translation process at least doubles decod- ing time during inference because more than one pivot language may be required to translate from the source to the target language. Additionally, transla- tion errors compound in a pipeline. Zero-shot NMT approaches are inspirited from multilingual NMT (multi-NMT) systems that use only one encoder and one decoder to represent mul- tiple languages in the same vector space, hence it should be possible to take advantage of data from high-resource language pairs to improve the trans- lation of low-resource language pairs. (Ha et al., 2016; Johnson et al., 2017) showed that the zero- shot systems are able to generate reasonable out- put at the target language by adding the desired out- put language’s language tag at the beginning of the source sentence. Note that there is no direct parallel data between the source and target languages dur- ing training. However, the performance of these ap- proaches is still poor when the source and target lan- guages are unrelated or the observed language pairs are not enough to capture the relation of unseen lan- guage pairs. Similar to the above approaches, …

Content truncated to 500 words