Reference Language based Unsupervised Neural Machine Translation

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4151–4162 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 4151 Reference Language based Unsupervised Neural Machine Translation Zuchao Li1,2,3, Hai Zhao1,2,3,∗, Rui Wang4,∗, Masao Utiyama4, and Eiichiro Sumita4 1Department of Computer Science and Engineering, Shanghai Jiao Tong University (SJTU) 2Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, China 3MoE Key Lab of Artificial Intelligence, AI Institude, Shanghai Jiao Tong University, China 4National Institute of Information and Communications Technology (NICT), Kyoto, Japan charlee@sjtu.edu.cn, zhaohai@cs,sjtu.edu.cn, {wangrui, mutiyama, eiichiro.sumita}@nict.go.jp Abstract Exploiting a common language as an auxiliary for better translation has a long tradition in machine translation and lets supervised learning-based machine translation enjoy the enhancement delivered by the well-used pivot language in the absence of a source language to target language parallel corpus. The rise of unsupervised neural machine translation (UNMT) almost completely relieves the parallel corpus curse, though UNMT is still subject to unsatisfactory performance due to the vagueness of the clues available for its core back-translation training. Further enriching the idea of pivot translation by extending the use of parallel corpora beyond the source-target paradigm, we propose a new reference language-based framework for UNMT, RUNMT, in which the reference language only shares a parallel corpus with the source, but this corpus still indicates a signal clear enough to help the reconstruction train- ing of UNMT through a proposed reference agreement mechanism. Experimental results show that our methods improve the quality of UNMT over that of a strong baseline that uses only one auxiliary language, demonstrating the usefulness of the proposed reference language-based UNMT and establishing a good start for the community. 1 Introduction Recently, the application of neural machine translation (NMT) (Sutskever et al., 2014; ∗ Corresponding authors. This paper was partially supported by National Key Research and Development Program of China (No. 2017YFB0304100), Key Projects of National Natural Science Foundation of China (U1836222 and 61733011), Huawei-SJTU Long Term AI Project, Cutting- edge Machine Reading Comprehension and Language Model. Rui Wang was partially supported by JSPS grant-in-aid for early-career scientists (19K20354): “Unsupervised Neural Machine Translation in Universal Scenarios” and NICT tenure- track researcher startup fund “Toward Intelligent Machine Translation”. w/ parallel corpus w/o parallel corpus translation direction S R T Reference language-based UNMT (c) Pivot Supervised NMT (a) MUNMT T S P T S P (b) Figure 1: Schemas of (a) pivot supervised NMT, (b) MUNMT, (c) our proposed RUNMT, where S stands for source language, T for target language, P for pivot language in pivot translation, and R for the reference language in RUNMT. Bahdanau et al., 2015) to standard benchmarks has achieved great success (Wu et al., 2016; Gehring et al., 2017; Vaswani et al., 2017) because of advances in deep learning and the availability of large-scale parallel corpora; however, the applicability of MT systems is limited because of their reliance on large parallel corpora for the majority of language pairs. In real-world situations, the majority of …

Content truncated to 500 words