Convolution-Enhanced Bilingual Recursive Neural Network for Bilingual Semantic Modeling

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3071–3081, Osaka, Japan, December 11-17 2016. Convolution-Enhanced Bilingual Recursive Neural Network for Bilingual Semantic Modeling Jinsong Su1, Biao Zhang1, Deyi Xiong2∗, Ruochen Li1, Jianmin Yin3 Xiamen University, Xiamen, China 3610051 Soochow University, Suzhou, China 2150062 Weifang Beida Jade Bird Huaguang Information Technology Co., Ltd, Weifang, China 2612053 jssu@xmu.edu.cn, zb@stu.xmu.edu.cn dyxiong@suda.edu.cn, lrc n@stu.xmu.edu.cn, jimyin@vip.sina.com Abstract Estimating similarities at different levels of linguistic units, such as words, sub-phrases and phrases, is helpful for measuring semantic similarity of an entire bilingual phrase. In this paper, we propose a convolution-enhanced bilingual recursive neural network (ConvBRNN), which not only exploits word alignments to guide the generation of phrase structures but also integrates multiple-level information of the generated phrase structures into bilingual semantic modeling. In order to accurately learn the semantic hierarchy of a bilingual phrase, we develop a recursive neural network to constrain the learned bilingual phrase structures to be consistent with word alignments. Upon the generated source and target phrase structures, we stack a convolutional neural network to integrate vector representations of linguistic units on the structures into bilin- gual phrase embeddings. After that, we fully incorporate information of different linguistic units into a bilinear semantic similarity model. We introduce two max-margin losses to train the Con- vBRNN model: one for the phrase structure inference and the other for the semantic similarity model. Experiments on NIST Chinese-English translation tasks demonstrate the high quality of the generated bilingual phrase structures with respect to word alignments and the effectiveness of learned semantic similarities on machine translation. 1 Introduction Recently, adapting deep neural networks to statistical machine translation (SMT) is of growing interest due to their superior capacity against conventional lexical models in feature learning and representation (Yang et al., 2013; Liu et al., 2013; Li et al., 2013; Devlin et al., 2014; Liu et al., 2014; Setiawan et al., 2015). As phrases are the basic translation units in many SMT systems, one line of research among these studies is to learn the semantic similarity of bilingual phrases for translation selection in SMT (Zhang et al., 2014a; Gao et al., 2014; Cho et al., 2014; Su et al., 2015; Hu et al., 2015). Typically, these bilingual semantic similarity models learn source and target phrase representations with some bilingual constraints (Gao et al., 2014; Hu et al., 2015; Zhang et al., 2014a). In spite of their success, they often suffer from two problems. Firstly, it is difficult for them to recover the semantic hierarchy (binary tree structure) of a bilingual phrase. In this respect, Su et al. (2015) improve tree con- struction by incorporating word alignments into their objective function. Unfortunately, they still employ the recursive autoencoder (RAE) as the underlying model to build tree structures of phrases according to the minimum reconstruction error. As a result, word alignments are not fully exploited for phrase structure generation. Secondly, the previous bilingual semantic similarity models are incapable of lever- aging representations at different levels of …

Content truncated to 500 words