Unsupervised Pivot Translation for Distant Languages

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 175–183 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 175 Unsupervised Pivot Translation for Distant Languages Yichong Leng∗ University of Science and Technology of China lyc123go@mail.ustc.edu.cn Xu Tan Microsoft Research xuta@microsoft.com Tao Qin† Microsoft Research taoqin@microsoft.com Xiang-Yang Li† University of Science and Technology of China xiangyangli@ustc.edu.cn Tie-Yan Liu Microsoft Research tyliu@microsoft.com Abstract Unsupervised neural machine translation (NMT) has attracted a lot of attention re- cently. While state-of-the-art methods for unsupervised translation usually perform well between similar languages (e.g., English- German translation), they perform poorly between distant languages, because unsu- pervised alignment does not work well for distant languages. In this work, we introduce unsupervised pivot translation for distant languages, which translates a language to a distant language through multiple hops, and the unsupervised translation on each hop is relatively easier than the original direct translation. We propose a learning to route (LTR) method to choose the translation path between the source and target languages. LTR is trained on language pairs whose best translation path is available and is applied to the unseen language pairs for path selection. Experiments on 20 languages and 294 distant language pairs demonstrate the advantages of the unsupervised pivot translation for distant languages, as well as the effectiveness of the proposed LTR for path selection. Specifically, in the best case, LTR achieves an improvement of 5.58 BLEU points over the conventional direct unsupervised method. 1 Introduction Unsupervised neural machine translation (NMT) (Artetxe et al., 2017b; Lample et al., 2017, 2018), which uses only monolingual sentences for translation, is of great importance ∗ The work was done when the first author was an intern at Microsoft Research Asia. † Corresponding author for zero-resource language pairs. Unsupervised translation relies on unsupervised cross-lingual word alignment or sentence alignment (Conneau et al., 2017; Artetxe et al., 2017a), where word embedding mapping (Artetxe et al., 2017b; Lam- ple et al., 2017) and vocabulary sharing (Lample et al., 2018) are used for word alignment, and encoder/decoder weight sharing (Artetxe et al., 2017b; Lample et al., 2018) and adversarial train- ing (Lample et al., 2017) are used for sentence alignment. Unsupervised cross-lingual alignment works reasonably well for a pair of similar languages, such as English-German or Portuguese-Galician, since they have similar lexica and syntax and share the same alphabets and language branch. However, the alignment between a pair of dis- tant languages, which are not in the same lan- guage branch1, such as Danish-Galician is chal- lenging. As a consequence, unsupervised transla- tion between distant languages is usually of lower quality. For example, the unsupervised NMT model achieves 23.43 BLEU score on Portuguese- Galician translation, while just 6.56 on Danish- Galician translation according to our experiments. In this work, we focus on unsupervised translation of distant languages. We observe that two distant languages can be linked through multiple intermediate hops where unsupervised translation of two languages on each 1In this work, we use language branch …

Content truncated to 500 words