Online Polylingual Topic Models for Fast Document Translation Detection

Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 252–261, Sofia, Bulgaria, August 8-9, 2013 c⃝2013 Association for Computational Linguistics Online Polylingual Topic Models for Fast Document Translation Detection Kriste Krstovski School of Computer Science University of Massachusetts Amherst Amherst, MA, 01003 kriste@cs.umass.edu David A. Smith School of Computer Science University of Massachusetts Amherst Amherst, MA, 01003 dasmith@cs.umass.edu Abstract Many tasks in NLP and IR require ef- ficient document similarity computations. Beyond their common application to ex- ploratory data analysis, latent variable topic models have been used to represent text in a low-dimensional space, indepen- dent of vocabulary, where documents may be compared. This paper focuses on the task of searching a large multilingual col- lection for pairs of documents that are translations of each other. We present (1) efficient, online inference for repre- senting documents in several languages in a common topic space and (2) fast ap- proximations for finding near neighbors in the probability simplex. Empirical evalu- ations show that these methods are as ac- curate as—and significantly faster than— Gibbs sampling and brute-force all-pairs search. 1 Introduction Statistical topic models, such as latent Dirich- let allocation (LDA) (Blei et al., 2003), have proven to be highly effective at discovering hid- den structure in document collections (Hall et al., 2008, e.g.). Often, these models facilitate ex- ploratory data analysis, by revealing which col- locations of terms are favored in different kinds of documents or which terms and topics rise and fall over time (Blei and Lafferty, 2006; Wang and McCallum, 2006). One of the greatest advan- tages in using topic models to analyze and process large document collections is their ability to rep- resent documents as probability distributions over a small number of topics, thereby mapping doc- uments into a low-dimensional latent space—the T-dimensional probability simplex, where T is the number of topics. A document, represented by some point in this simplex, is said to have a par- ticular “topic distribution”. Representing documents as points in a low- dimensional shared latent space abstracts away from the specific words used in each document, thereby facilitating the analysis of relationships between documents written using different vocab- ularies. For instance, topic models have been used to identify scientific communities working on re- lated problems in different disciplines, e.g., work on cancer funded by multiple Institutes within the NIH (Talley et al., 2011). While vocabulary mis- match occurs within the realm of one language, naturally this mismatch occurs across different languages. Therefore, mapping documents in dif- ferent languages into a common latent topic space can be of great benefit when detecting document translation pairs (Mimno et al., 2009; Platt et al., 2010). Aside from the benefits that it offers in the task of detecting document translation pairs, topic models offer potential benefits to the task of creat- ing translation lexica, aligning passages, etc. The process of discovering relationship be- tween documents using topic models involves: (1) representing documents in the latent space by in- ferring their topic distributions and (2) comparing …

Content truncated to 500 words