Synchronously Generating Two Languages with Interactive Decoding

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3350–3355, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3350 Synchronously Generating Two Languages with Interactive Decoding Yining Wang1,2, Jiajun Zhang1,2∗, Long Zhou1,2 Yuchen Liu1,2 and Chengqing Zong1,2,3 1National Laboratory of Pattern Recognition, CASIA, Beijing, China 2University of Chinese Academy of Sciences, Beijing, China 3CAS Center for Excellence in Brain Science and Intelligence Technology, Beijing, China {yining.wang, jjzhang, long.zhou}@nlpr.ia.ac.cn {yuchen.liu, cqzong}@nlpr.ia.ac.cn Abstract In this paper, we introduce a novel interactive approach to translate a source language into two different languages simultaneously and in- teractively. Specifically, the generation of one language relies on not only previously gener- ated outputs by itself, but also the outputs pre- dicted in the other language. Experimental re- sults on IWSLT and WMT datasets demon- strate that our method can obtain significant improvements over both conventional Neural Machine Translation (NMT) model and multi- lingual NMT model. 1 Introduction Neural Machine Translation (NMT) based on the encoder-decoder framework has significantly im- proved translation quality due to its powerful end- to-end modeling (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017; Gehring et al., 2017; Hassan et al., 2018; Zhang and Zong, 2015). This paradigm facilitates the development of mul- tilingual NMT (Dong et al., 2015; Luong et al., 2016; Johnson et al., 2017; Ha et al., 2016; Firat et al., 2016; Lakew et al., 2017; Tan et al., 2019; Wang et al., 2019), which handles multiple lan- guage pairs in one model, with the benefit of sim- plifying offline model training and easing online maintenance cost. Although multilingual NMT attempts to utilize the complementary information of different lan- guages (Lu et al., 2018; Neubig and Hu, 2018; Pla- tanios et al., 2018; Wang et al., 2018), all of the models handle one language pair at each moment for both training and testing. However, we find that the generation process of different target lan- guages can help each other. For example in Figure 1, when decoding the Chinese word “书” meaning “book” at step t = 5, the predicted Japanese word ∗Jiajun Zhang is the corresponding author. 我 I 买了 bought 一本 pad 一冊 a の 本 book 買い bought t=1 t=2 t=4 t=5 今天 today 今日 today t=3 书 book を I bought a book today. English Chinese Japanese pad 一本 a ました Figure 1: An example of an English sentence trans- lated into Chinese and Japaneses sentences, in which two targets can interact with each other. “本” with same meaning can provide the context at step t = 4. The reason is that the sentence struc- ture between the two languages is different. It is subject-verb-object in Chinese while it is subject- object-verb in Japanese. Moreover, we find that two languages are complementary, and if decoders belonging to two different languages can interact with each other, the quality of translation will be improved. In this work, we …

Content truncated to 500 words