Translation Model Based Cross-Lingual Language Model Adaptation: from Word Models to Phrase Models

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 512–522, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics Translation Model Based Cross-Lingual Language Model Adaptation: from Word Models to Phrase Models Shixiang Lu, Wei Wei, Xiaoyin Fu, and Bo Xu Interactive Digital Media Technology Research Center Institute of Automation, Chinese Academy of Sciences 95 Zhongguancun East Road, Haidian District, Beijing 100190, China {shixiang.lu,wei.wei.media,xiaoyin.fu,xubo}@ia.ac.cn Abstract In this paper, we propose a novel translation model (TM) based cross-lingual data selec- tion model for language model (LM) adapta- tion in statistical machine translation (SMT), from word models to phrase models. Given a source sentence in the translation task, this model directly estimates the probability that a sentence in the target LM training corpus is similar. Compared with the traditional ap- proaches which utilize the first pass translation hypotheses, cross-lingual data selection mod- el avoids the problem of noisy proliferation. Furthermore, phrase TM based cross-lingual data selection model is more effective than the traditional approaches based on bag-of- words models and word-based TM, because it captures contextual information in model- ing the selection of phrase as a whole. Ex- periments conducted on large-scale data set- s demonstrate that our approach significantly outperforms the state-of-the-art approaches on both LM perplexity and SMT performance. 1 Introduction Language model (LM) plays a critical role in sta- tistical machine translation (SMT). It seems to be a universal truth that LM performance can always be improved by using more training data (Brants et al., 2007), but only if the training data is reason- ably well-matched with the desired output (Moore and Lewis, 2010). It is also obvious that among the large training data the topics or domains of discus- sion will change (Eck et al., 2004), which causes the mismatch problems with the translation task. For this reason, most researchers preferred to select sim- ilar training data from the large training corpus in the past few years (Eck et al., 2004; Zhao et al., 2004; Kim, 2005; Masskey and Sethy, 2010; Axelrod et al., 2011). This would empirically provide more ac- curate lexical probabilities, and thus better match the translation task at hand (Axelrod et al., 2011). Many previous data selection approaches for LM adaptation in SMT depend on the first pass transla- tion hypotheses (Eck et al., 2004; Zhao et al., 2004; Kim, 2005; Masskey and Sethy, 2010), they selec- t the sentences which are similar to the translation hypotheses. These schemes are overall limited by the quality of the translation hypotheses (Tam et al., 2007 and 2008), and better initial translation hy- potheses lead to better selected sentences (Zhao et al., 2004). However, while SMT has achieved a great deal of development in recent years, the trans- lation hypotheses are still far from perfect (Wei and Pal, 2010), which have many noisy data. The noisy translation hypotheses mislead data selection pro- cess (Xu et al., 2001; Tam et al., 2006 and 2007; Wei and …

Content truncated to 500 words