Knowledge-Based Semantic Embedding for Machine Translation

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2245–2254, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Knowledge-Based Semantic Embedding for Machine Translation Chen Shi†∗ Shujie Liu‡ Shuo Ren‡ Shi Feng§ Mu Li‡ Ming Zhou‡ Xu Sun† Houfeng Wang†¶ †MOE Key Lab of Computational Linguistics, Peking University ‡Microsoft Research Asia §Shanghai Jiao Tong University ¶Collaborative Innovation Center for Language Ability {shichen, xusun, wanghf}@pku.edu.cn sjtufs@gmail.com {shujliu, v-shuren, muli, mingzhou}@microsoft.com Abstract In this paper, with the help of knowl- edge base, we build and formulate a se- mantic space to connect the source and target languages, and apply it to the sequence-to-sequence framework to pro- pose a Knowledge-Based Semantic Em- bedding (KBSE) method. In our KB- SE method, the source sentence is firstly mapped into a knowledge based seman- tic space, and the target sentence is gen- erated using a recurrent neural network with the internal meaning preserved. Ex- periments are conducted on two transla- tion tasks, the electric business data and movie data, and the results show that our proposed method can achieve outstanding performance, compared with both the tra- ditional SMT methods and the existing encoder-decoder models. 1 Introduction Deep neural network based machine translation, such as sequence-to-sequence (S2S) model (Cho et al., 2014; Sutskever et al., 2014), try to learn translation relation in a continuous vector space. As shown in Figure 1, the S2S framework contains two parts: an encoder and a decoder. To compress a variable-length source sentence into a fixed-size vector, with a recurrent neural network (RNN), an encoder reads words one by one and generates a sequence of hidden vectors. By reading all the source words, the final hidden vector should con- tain the information of source sentence, and it is called the context vector. Based on the context vector, another RNN-based neural network is used to generate the target sentence. ∗This work was done while the first author was visiting Microsoft Research. yT’ y2 y1 X1 X2 XT Decoder Encoder c 给 我 推荐 个 4G 手机 吧 ，最 好 白 的 ，屏幕 要 大。 I want a white 4G cellphone with a big screen. Figure 1: An illustration of the RNN-based neural network model for Chinese-to-English machine translation The context vector plays a key role in the con- nection of source and target language spaces, and it should contain all the internal meaning extracted from source sentence, based on which, the decoder can generate the target sentence keeping the mean- ing unchanged. To extract the internal meaning and generate the target sentence, S2S framework usually needs large number of parameters, and a big bilingual corpus is acquired to train them. In many cases, the internal meaning is not easy to learn, especially when the language is informal. For the same intention, there are various expres- sions with very different surface string, which ag- gravates the difficulty of internal meaning extrac- tion. As shown in Table 1, there are three different expressions for a same intention, a customer …

Content truncated to 500 words