A Context-Aware Topic Model for Statistical Machine Translation

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 229–238, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics A Context-Aware Topic Model for Statistical Machine Translation Jinsong Su1, Deyi Xiong2∗, Yang Liu3, Xianpei Han4, Hongyu Lin1, Junfeng Yao1, Min Zhang2 Xiamen University, Xiamen, China1 Soochow University, Suzhou, China2 Tsinghua University, Beijing, China3 Institute of Software, Chinese Academy of Sciences, Beijing, China4 {jssu, hylin, yao0010}@xmu.edu.cn {dyxiong, minzhang}@suda.edu.cn liuyang2011@tsinghua.edu.cn xianpei@nfs.iscas.ac.cn Abstract Lexical selection is crucial for statistical ma- chine translation. Previous studies separately exploit sentence-level contexts and document- level topics for lexical selection, neglecting their correlations. In this paper, we propose a context-aware topic model for lexical selec- tion, which not only models local contexts and global topics but also captures their correla- tions. The model uses target-side translations as hidden variables to connect document top- ics and source-side local contextual words. In order to learn hidden variables and distribu- tions from data, we introduce a Gibbs sam- pling algorithm for statistical estimation and inference. A new translation probability based on distributions learned by the model is inte- grated into a translation system for lexical se- lection. Experiment results on NIST Chinese- English test sets demonstrate that 1) our model significantly outperforms previous lexical se- lection methods and 2) modeling correlations between local words and global topics can fur- ther improve translation quality. 1 Introduction Lexical selection is a very important task in statis- tical machine translation (SMT). Given a sentence in the source language, lexical selection statistically predicts translations for source words, based on vari- ous translation knowledge. Most conventional SMT systems (Koehn et al., 2003; Galley et al., 2006; Chiang, 2007) exploit very limited context informa- tion contained in bilingual rules for lexical selection. ∗Corresponding author. {stance, attitude ...}lìchǎng duì gāi wèntí zhōngguó bǎochí zhōnglì lìchǎng [Economy topic, Politics topic ...] {problem, issue ...}wèntí Figure 1: A Chinese-English translation example to il- lustrate the effect of local contexts and global topics as well as their correlations on lexical selection. Each black line indicates a set of translation candidates for a Chinese content word (within a dotted box). Green lines point to translations that are favored by local contexts while blue lines show bidirectional associations between global top- ics and their consistent target-side translations. Previous studies that explore richer information for lexical selection can be divided into two categories: 1) incorporating sentence-level contexts (Chan et al., 2007; Carpuat and Wu, 2007; Hasan et al., 2008; Mauser et al., 2009; He et al., 2008; Shen et al., 2009) or 2) integrating document-level topics (Xi- ao et al., 2011; Ture et al., 2012; Xiao et al., 2012; Eidelman et al., 2012; Hewavitharana et al., 2013; Xiong et al., 2013; Hasler et al., 2014a; Hasler et al., 2014b) into SMT. The methods in these two strands have shown their effectiveness on lexical selection. However, correlations between sentence- and document-level contexts have never been explored before. It is clear …

Content truncated to 500 words