Topic Adaptation for Lecture Translation through Bilingual Latent Semantic Models

Proceedings of the 6th Workshop on Statistical Machine Translation, pages 294–302, Edinburgh, Scotland, UK, July 30–31, 2011. c⃝2011 Association for Computational Linguistics Topic Adaptation for Lecture Translation through Bilingual Latent Semantic Models Nick Ruiz∗ Free University of Bozen-Bolzano Bolzano, Italy nicruiz@fbk.eu Marcello Federico FBK-irst Fondazione Bruno Kessler Trento, Italy federico@fbk.eu Abstract This work presents a simplified approach to bilingual topic modeling for language model adaptation by combining text in the source and target language into very short documents and performing Probabilistic Latent Semantic Analysis (PLSA) during model training. Dur- ing inference, documents containing only the source language can be used to infer a full topic-word distribution on all words in the tar- get language’s vocabulary, from which we per- form Minimum Discrimination Information (MDI) adaptation on a background language model (LM). We apply our approach on the English-French IWSLT 2010 TED Talk exer- cise, and report a 15% reduction in perplexity and relative BLEU and NIST improvements of 3% and 2.4%, respectively over a baseline only using a 5-gram background LM over the entire translation task. Our topic modeling ap- proach is simpler to construct than its counter- parts. 1 Introduction Adaptation is usually applied to reduce the per- formance drop of Statistical Machine Translation (SMT) systems when translating documents that de- viate from training and tuning conditions. In this paper, we focus primarily on language model (LM) adaptation. In SMT, LMs are used to promote fluent translations. As probabilistic models of sequences of words, language models guide the selection and ordering of phrases in translation. With respect to ∗This work was carried out during an internship period at Fondazione Bruno Kessler. LM training, LM adaptation for SMT tries to im- prove an existing LM by using smaller amounts of texts. When adaptation data represents the trans- lation task domain one generally refers to domain adaptation, while when they just represent the con- tent of the single document to be translated one typ- ically refers to topic adaptation. We propose a cross-language topic adaptation method, enabling the adaptation of a LM based on the topic distribution of the source document dur- ing translation. We train a latent semantic topic model on a collection of bilingual documents, in which each document contains both the source and target language. During inference, a latent topic dis- tribution of words across both the source and tar- get languages is inferred from a source document to be translated. After inference, we remove all source language words from the topic-word distribu- tions and construct a unigram language model which is used to adapt our background LM via Minimum Discrimination Information (MDI) estimation (Fed- erico, 1999, 2002; Kneser et al., 1997). We organize the paper as follows: In Section 2, we discuss relevant previous work. In Section 3, we review topic modeling. In Section 4, we review MDI adaptation. In Section 5, we describe our new bilin- gual topic modeling based adaptation technique. In Section 6, we report adaptation experiments, fol- lowed by conclusions and future work …

Content truncated to 500 words