Bilingual-LSA Based LM Adaptation for Spoken Language Translation

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 520–527, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Bilingual-LSA Based LM Adaptation for Spoken Language Translation Yik-Cheung Tam and Ian Lane and Tanja Schultz InterACT, Language Technologies Institute Carnegie Mellon University Pittsburgh, PA 15213 {yct,ian.lane,tanja}@cs.cmu.edu Abstract We propose a novel approach to crosslingual language model (LM) adaptation based on bilingual Latent Semantic Analysis (bLSA). A bLSA model is introduced which enables latent topic distributions to be efficiently transferred across languages by enforcing a one-to-one topic correspondence during training. Using the proposed bLSA frame- work crosslingual LM adaptation can be per- formed by, first, inferring the topic poste- rior distribution of the source text and then applying the inferred distribution to the tar- get language N-gram LM via marginal adap- tation. The proposed framework also en- ables rapid bootstrapping of LSA models for new languages based on a source LSA model from another language. On Chinese to English speech and text translation the proposed bLSA framework successfully re- duced word perplexity of the English LM by over 27% for a unigram LM and up to 13.6% for a 4-gram LM. Furthermore, the pro- posed approach consistently improved ma- chine translation quality on both speech and text based adaptation. 1 Introduction Language model adaptation is crucial to numerous speech and translation tasks as it enables higher- level contextual information to be effectively incor- porated into a background LM improving recogni- tion or translation performance. One approach is to employ Latent Semantic Analysis (LSA) to cap- ture in-domain word unigram distributions which are then integrated into the background N-gram LM. This approach has been successfully applied in automatic speech recognition (ASR) (Tam and Schultz, 2006) using the Latent Dirichlet Alloca- tion (LDA) (Blei et al., 2003). The LDA model can be viewed as a Bayesian topic mixture model with the topic mixture weights drawn from a Dirichlet distribution. For LM adaptation, the topic mixture weights are estimated based on in-domain adapta- tion text (e.g. ASR hypotheses). The adapted mix- ture weights are then used to interpolate a topic- dependent unigram LM, which is finally integrated into the background N-gram LM using marginal adaptation (Kneser et al., 1997) In this paper, we propose a framework to per- form LM adaptation across languages, enabling the adaptation of a LM from one language based on the adaptation text of another language. In statistical machine translation (SMT), one approach is to ap- ply LM adaptation on the target language based on an initial translation of input references (Kim and Khudanpur, 2003; Paulik et al., 2005). This scheme is limited by the coverage of the translation model, and overall by the quality of translation. Since this approach only allows to apply LM adaptation af- ter translation, available knowledge cannot be ap- plied to extend the coverage. We propose a bilingual LSA model (bLSA) for crosslingual LM adaptation that can be applied before translation. The bLSA model consists of two LSA models: one …

Content truncated to 500 words