A Topic-Triggered Language Model for Statistical Machine Translation

International Joint Conference on Natural Language Processing, pages 447–454, Nagoya, Japan, 14-18 October 2013. A Topic-Triggered Language Model for Statistical Machine Translation Heng Yu†, Jinsong Su∗, Yajuan l¨u†, Qun Liu ‡ † †Institute of Computing Technology, Chinese Academy of Sciences ∗ Software School of Xiamen University ‡ Centre for Next Generation Localisation Faculty of Engineering and Computing, Dublin City University {yuheng,lvyajuan,liuqun}@ict.ac.cn jssu@xmu.edu.cn qliu@computing.dcu.ie Abstract Language model is an essential part in sta- tistical machine translation, but traditional n-gram language models can only capture a limited local context in the translated sentence, thus lacking the global informa- tion for prediction. This paper describes a novel topic-triggered language model, which takes into account the topical con- text by estimating the n-gram probabil- ity under the given topics and online ad- justs language model score according to different topic distributions. Experimental results show that our method provides a average improvement of +0.76 Bleuon NIST Chinese-to-English translation task and a reduction in word perplexity of the test-document. 1 Introduction Language model (LM) measures the fluency of translation outputs (Brown et al., 1993), and plays an important role in statistical machine translation (SMT). Traditional language model predicts the next word conditioning only on the preceding n−1 words, thus ignores syntactic structures in the sen- tence and global information over the document. One direct approach to handle this problem is to explore sentence-level context, such as syntax- based language model for reranking (Charniak et al., 2003), and dependency language model for String-to-Dependency model (Shen et al., 2008). But these methods are still not robust enough to handle the polysemy and domain changes, as they lack the global-context information. Another interesting line is to utilize informa- tion at document-level. Intuitively, different do- mains or topics have different n-gram probability distributions. Thus, we should take into account the topic information when we translate a doc- ument. Topic model has been learned in several parts of SMT, such as word-alignment (Zhao and Xing, 2006; Zhao and Xing, 2007; Gong et al., 2011), translation model (Xiao et al., 2012). All these works show that a particular translation of- ten appears in some specific topical context, so it is reasonable to enhance the prediction ability of language model by incorporating topical informa- tion. Tan et al. (2011) introduces a large scale dis- tributed composite language model incorporating document-level information. But they only focus on the target side and explore in n-best rerank- ing task which has a limited search space, while another promising application is taking account of topical information on both sides and integrate the LM into decoding to online select transla- tion hypotheses. However, the integration is not easy. Since the test-document can be from any topic, it is hard to dynamically estimate language model probability according to various topic dis- tributions. In this paper, we follow this line and introduce a novel topic-triggered language model. We first es- timate the topic distribution for each document in training data, and assign those topic probabilities to each sentence. With target-side …

Content truncated to 500 words