Unsupervised Neural Machine Translation with Future Rewarding

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 281–290 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 281 Unsupervised Neural Machine Translation with Future Rewarding Xiangpeng Wei1,2, Yue Hu1,2∗, Luxi Xing1,2, Li Gao3 1Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China 2School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China 3Platform & Content Group, Tencent, Beijing, China {weixiangpeng, huyue, xingluxi}@iie.ac.cn leolgao@tencent.com Abstract In this paper, we alleviate the local optimality of back-translation by learning a policy (takes the form of an encoder-decoder and is defined by its parameters) with future rewarding under the reinforcement learning framework, which aims to optimize the global word predictions for unsupervised neural machine translation. To this end, we design a novel reward function to characterize high-quality translations from two aspects: n-gram matching and semantic adequacy. The n-gram matching is defined as an alternative for the discrete BLEU met- ric, and the semantic adequacy is used to mea- sure the adequacy of conveying the meaning of the source sentence to the target. During training, our model strives for earning higher rewards by learning to produce grammatically more accurate and semantically more ade- quate translations. Besides, a variational infer- ence network (VIN) is proposed to constrain the corresponding sentences in two languages have the same or similar latent semantic code. On the widely used WMT’14 English-French, WMT’16 English-German and NIST Chinese- to-English benchmarks, our models respec- tively obtain 27.59/27.15, 19.65/23.42 and 22.40 BLEU points without using any labeled data, demonstrating consistent improvements over previous unsupervised NMT models. 1 Introduction Neural Machine Translation (Sutskever et al., 2014; Bahdanau et al., 2015) directly models the entire translation process through training an encoder-decoder model that has achieved remark- able performance (Wu et al., 2016; Gehring et al., 2017; Vaswani et al., 2017) when provided with massive amounts of parallel corpora. However, the lack of large-scale parallel data is a serious prob- lem for the vast majority of language pairs. ∗Corresponding Author. As a result, several works have recently tried to get rid of the dependence on parallel corpora us- ing unsupervised setting, in which the NMT model only has access to two independent monolingual corpora with one for each language (Lample et al., 2018a; Artetxe et al., 2018b; Yang et al., 2018). Among these works, the encoder and decoder act as a standard auto-encoder (AE) that are trained to reconstruct the inputs from their noised versions. Due to the lack of cross-language signals, unsu- pervised NMT usually requires pseudo parallel data generated with the back-translation method for achieving the final goal of translating between source and target languages. Back-translation typically uses beam search (Sennrich et al., 2016a) or just greedy search (Lample et al., 2018a,b) to generate synthetic sentences. Both are approximate al- gorithms to identify the maximum a posteriori (MAP) output, i.e. the sentence with the highest estimated probability given an input. Although back-translation with MAP prediction has been proved to be successful, it suffers from several apparent …

Content truncated to 500 words