Improve Statistical Machine Translation with Context-Sensitive Bilingual Semantic Embedding Model

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 142–146, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Improve Statistical Machine Translation with Context-Sensitive Bilingual Semantic Embedding Model Haiyang Wu1 Daxiang Dong1 Wei He1 Xiaoguang Hu1 Dianhai Yu1 Hua Wu1 Haifeng Wang1 Ting Liu2 1 Baidu Inc., No. 10, Shangdi 10th Street, Beijing, 100085, China 2 Harbin Institute of Technology, Harbin, China wuhaiyang,dongdaxiang,hewei,huxiaoguang,yudianhai, wu hua,wanghaifeng@baidu.com tliu@ir.hit.edu.cn Abstract We investigate how to improve bilingual embedding which has been successfully used as a feature in phrase-based sta- tistical machine translation (SMT). De- spite bilingual embedding’s success, the contextual information, which is of criti- cal importance to translation quality, was ignored in previous work. To employ the contextual information, we propose a simple and memory-efficient model for learning bilingual embedding, taking both the source phrase and context around the phrase into account. Bilingual translation scores generated from our proposed bilin- gual embedding model are used as features in our SMT system. Experimental results show that the proposed method achieves significant improvements on large-scale Chinese-English translation task. 1 Introduction In Statistical Machine Translation (SMT) sys- tem, it is difficult to determine the translation of some phrases that have ambiguous meanings.For example, the phrase “结果 jieguo” can be trans- lated to either “results”, “eventually” or “fruit”, depending on the context around it. There are two reasons for the problem: First, the length of phrase pairs is restricted due to the limitation of model size and training data. Another reason is that SMT systems often fail to use contextual information in source sentence, therefore, phrase sense disam- biguation highly depends on the language model which is trained only on target corpus. To solve this problem, we present to learn context-sensitive bilingual semantic embedding. Our methodology is to train a supervised model where labels are automatically generated from phrase-pairs. For each source phrase, the aligned target phrase is marked as the positive label whereas other phrases in our phrase table are treated as negative labels. Different from previ- ous work in bilingual embedding learning(Zou et al., 2013; Gao et al., 2014), our framework is a supervised model that utilizes contextual informa- tion in source sentence as features and make use of phrase pairs as weak labels. Bilingual seman- tic embeddings are trained automatically from our supervised learning task. Our learned bilingual semantic embedding model is used to measure the similarity of phrase pairs which is treated as a feature in decoding. We integrate our learned model into a phrase-based translation system and experimental results indi- cate that our system significantly outperform the baseline system. On the NIST08 Chinese-English translation task, we obtained 0.68 BLEU improve- ment. We also test our proposed method on much larger web dataset and obtain 0.49 BLEU im- provement against the baseline. 2 Related Work Using vectors to represent word meanings is the essence of vector space models (VSM). The representations capture words’ semantic and syn- tactic information which can be used to measure semantic similarities …

Content truncated to 500 words