Learning bilingual distributed phrase represenations for statistical machine translation

Learning Bilingual Distributed Phrase Representations for Statistical Machine Translation Chaochao Wang chaochaowang@foxmail.com Deyi Xiong ∗ dyxiong@suda.edu.cn Min Zhang minzhang@suda.edu.cn Soochow University, Suzhou, China Chunyu KIT ctckit@cityu.edu.hk Department of Linguistics and Translation, City University of Hong Kong Abstract Following the idea of using distributed semantic representations to facilitate the computation of semantic similarity between translation equivalents, we propose a novel framework to learn bilingual distributed phrase representations for machine translation. We first induce vector representations for words in the source and target language respectively, in their own semantic space. These word vectors are then used to create phrase representations via composition meth- ods. In order to compute semantic similarity of phrase pairs in the same semantic space, we project phrase representations from the source-side semantic space onto the target-side seman- tic space via a neural network that is able to conduct nonlinear transformation between the two spaces. We integrate the learned bilingual distributed phrase representations into a hierarchi- cal phrase-based translation system to validate the effectiveness of our proposed framework. Experiment results show that our method is able to significantly improve translation quality and outperform previous methods that only use word representations or linear semantic space transformation. 1 Introduction Distributional semantic models provide a means to represent meanings of words under the assumption that words occurring in similar contexts tend to have the same meanings (Har- ris,1968). Various distributed representations have been successfully applied to various mono- lingual natural language processing tasks, such as word sense discrimination (Clark and Pul- man,2007) and thesaurus compilation (Yang and Powers,2008). In this paper, we explore how to learn semantic representations of bilingual phrases, rather than monolingual words, in the context of statistical machine translation (SMT) to facilitate the computation of semantic sim- ilarity between translation equivalents at the phrase level. We also study whether semantic similarity scores calculated in terms of bilingual distributed phrase representations are comple- mentary to phrase translation probabilities estimated by the conventional counting method in SMT Koehn et al. (2003). Very recently we have witnessed some studies on learning bilingual distributed represen- tations for SMT. Mikolov et al. (2013b) train two neural network (NN) based models to learn word embeddings in the source and the target language, respectively, and then map the embed- dings from the source to the target language space using a transformation matrix that is learned ∗Corresponding author Proceedings of MT Summit XV, vol.1: MT Researchers' Track Miami, Oct 30 - Nov 3, 2015 | p. 32 by minimizing the mapping cost on all word pairs. Zou et al. (2013) introduce bilingual word embeddings into phrase-based machine translation: Word representations are first learned from language A via an NN-based model, word embeddings in the parallel language B are then ini- tialized according to A’s embeddings and word alignments between A and B, and the final word representations of B are obtained by a further training process that optimizes a combined ob- jective on bilingual data. Gao et al. (2013) extend distributional representations from the word level to …

Content truncated to 500 words