Polylingual Topic Models

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 880–889, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Polylingual Topic Models David Mimno Hanna M. Wallach Jason Naradowsky University of Massachusetts, Amherst Amherst, MA 01003 {mimno, wallach, narad, dasmith, mccallum}cs.umass.edu David A. Smith Andrew McCallum Abstract Topic models are a useful tool for analyz- ing large text collections, but have previ- ously been applied in only monolingual, or at most bilingual, contexts. Mean- while, massive collections of interlinked documents in dozens of languages, such as Wikipedia, are now widely available, calling for tools that can characterize con- tent in many languages. We introduce a polylingual topic model that discovers top- ics aligned across multiple languages. We explore the model’s characteristics using two large corpora, each with over ten dif- ferent languages, and demonstrate its use- fulness in supporting machine translation and tracking topic trends across languages. 1 Introduction Statistical topic models have emerged as an in- creasingly useful analysis tool for large text col- lections. Topic models have been used for analyz- ing topic trends in research literature (Mann et al., 2006; Hall et al., 2008), inferring captions for im- ages (Blei and Jordan, 2003), social network anal- ysis in email (McCallum et al., 2005), and expand- ing queries with topically related words in infor- mation retrieval (Wei and Croft, 2006). Much of this work, however, has occurred in monolingual contexts. In an increasingly connected world, the ability to access documents in many languages has become both a strategic asset and a personally en- riching experience. In this paper, we present the polylingual topic model (PLTM). We demonstrate its utility and explore its characteristics using two polylingual corpora: proceedings of the European parliament (in eleven languages) and a collection of Wikipedia articles (in twelve languages). There are many potential applications for polylingual topic models. Although research liter- ature is typically written in English, bibliographic databases often contain substantial quantities of work in other languages. To perform topic-based bibliometric analysis on these collections, it is necessary to have topic models that are aligned across languages. Such analysis could be sig- nificant in tracking international research trends, where language barriers slow the transfer of ideas. Previous work on bilingual topic modeling has focused on machine translation applications, which rely on sentence-aligned parallel transla- tions. However, the growth of the internet, and in particular Wikipedia, has made vast corpora of topically comparable texts—documents that are topically similar but are not direct translations of one another—considerably more abundant than ever before. We argue that topic modeling is both a useful and appropriate tool for leveraging correspondences between semantically compara- ble documents in multiple different languages. In this paper, we use two polylingual corpora to answer various critical questions related to polylingual topic models. We employ a set of di- rect translations, the EuroParl corpus, to evaluate whether PLTM can accurately infer topics when documents genuinely contain the same content. We also explore how the characteristics of dif- ferent languages affect …

Content truncated to 500 words