Polylingual Tree-Based Topic Models for Translation Domain Adaptation

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 1166–1176, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Polylingual Tree-Based Topic Models for Translation Domain Adaptation Yuening Hu† Computer Science University of Maryland ynhu@cs.umd.edu Ke Zhai† Computer Science University of Maryland zhaike@cs.umd.edu Vladimir Eidelman FiscalNote Inc. Washington DC vlad@fiscalnote.com Jordan Boyd-Graber iSchool and UMIACS University of Maryland jbg@umiacs.umd.edu Abstract Topic models, an unsupervised technique for inferring translation domains improve machine translation quality. However, pre- vious work uses only the source language and completely ignores the target language, which can disambiguate domains. We pro- pose new polylingual tree-based topic mod- els to extract domain knowledge that con- siders both source and target languages and derive three different inference schemes. We evaluate our model on a Chinese to En- glish translation task and obtain up to 1.2 BLEU improvement over strong baselines. 1 Introduction Probabilistic topic models (Blei and Lafferty, 2009), exemplified by latent Dirichlet alloca- tion (Blei et al., 2003, LDA), are one of the most popular statistical frameworks for navigating large unannotated document collections. Topic models discover—without any supervision—the primary themes presented in a dataset: the namesake topics. Topic models have two primary applications: to aid human exploration of corpora (Chang et al., 2009) or serve as a low-dimensional representa- tion for downstream applications. We focus on the second application, which has been fruitful for computer vision (Li Fei-Fei and Perona, 2005), computational biology (Perina et al., 2010), and information retrieval (Kataria et al., 2011). In particular, we use topic models to aid statisti- cal machine translation (Koehn, 2009, SMT). Mod- ern machine translation systems use millions of examples of translations to learn translation rules. These systems work best when the training corpus has consistent genre, register, and topic. Systems that are robust to systematic variation in the train- ing set are said to exhibit domain adaptation. † indicates equal contributions. As we review in Section 2, topic models are a promising solution for automatically discover- ing domains in machine translation corpora. How- ever, past work either relies solely on monolingual source-side models (Eidelman et al., 2012; Hasler et al., 2012; Su et al., 2012), or limited modeling of the target side (Xiao et al., 2012). In contrast, machine translation uses inherently multilingual data: an SMT system must translate a phrase or sen- tence from a source language to a different target language, so existing applications of topic mod- els (Eidelman et al., 2012) are wilfully ignoring available information on the target side that could aid domain discovery. This is not for a lack of multilingual topic mod- els. Topic models bridge the chasm between lan- guages using document connections (Mimno et al., 2009), dictionaries (Boyd-Graber and Resnik, 2010), and word alignments (Zhao and Xing, 2006). In Section 2, we review these models for discover- ing topics in multilingual datasets and discuss how they can improve SMT. However, no models combine multiple bridges between languages. In Section 3, we create a model—the polylingual …

Content truncated to 500 words