Reference Network for Neural Machine Translation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3002–3012 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3002 Reference Network for Neural Machine Translation Han Fu†‡ Chenghao Liu§ Jianling Sun†‡∗ †Zhejiang University, Hangzhou, China ‡Alibaba-Zhejiang University Joint Institute of Frontier Technologies, China §Singapore Management University, Singapore {11821003, sunjl}@zju.edu.cn twinsken@gmail.com Abstract Neural Machine Translation (NMT) has achieved notable success in recent years. Such a framework usually generates translations in isolation. In contrast, human translators often refer to reference data, either rephrasing the in- tricate sentence fragments with common terms in source language, or just accessing to the golden translation directly. In this paper, we propose a Reference Network to incorporate referring process into translation decoding of NMT. To construct a reference book, an intu- itive way is to store the detailed translation his- tory with extra memory, which is computation- ally expensive. Instead, we employ Local Co- ordinates Coding (LCC) to obtain global con- text vectors containing monolingual and bilin- gual contextual information for NMT decod- ing. Experimental results on Chinese-English and English-German tasks demonstrate that our proposed model is effective in improving the translation quality with lightweight com- putation cost. 1 Introduction Neural Machine Translation (NMT) has enjoyed impressive success in most large-scale transla- tion tasks (Kalchbrenner and Blunsom, 2013; Sutskever et al., 2014; Cho et al., 2014). Typical NMT model to date is a single end-to-end trained deep neural network that encodes the source sen- tence into a fixed-length vector and generates the words in the target sentence sequentially. The alignment relationship between source and target sentence is learned by the attention mechanism (Bahdanau et al., 2015; Luong et al., 2015). Though the framework has achieved significant success, one critical concern is that NMT gener- ates translations in isolation, which leads to trans- lation inconsistency and ambiguity arising from ∗Corresponding author: Jianling Sun. a single source sentence (Tu et al., 2018). Re- cently, there have been few attempts to model the semantic information across sentences. The basic ideas are to store a handful of previous source or target sentences with context vectors (Jean et al., 2017; Wang et al., 2017a) or memory compo- nents (Maruf and Haffari, 2018; Tu et al., 2018). However, these methods have several limitations. First, the very short view of the previous sentences (usually one or two sentence(s)) is not sufficient enough to catch long term dependencies across paragraphs and storing detailed translation history is computationally expensive. Second, in the real- world scenarios, input data of MT application is often isolated sentences, such as Google Trans- late, where no cross-sentence contexts are pro- vided. Moreover, translations generated by such document-level NMT models are not stable, ef- fected by the sentences surrounding the current one to translate. To address these limitations, we model the se- mantic information across sentences by mimick- ing the human translation process. In real scenar- ios, there will always be sentences or fragments that the translator can understand the meaning but cannot …

Content truncated to 500 words