Cross-Lingual Latent Topic Extraction

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1128–1137, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Cross-Lingual Latent Topic Extraction Duo Zhang University of Illinois at Urbana-Champaign dzhang22@cs.uiuc.edu Qiaozhu Mei University of Michigan qmei@umich.edu ChengXiang Zhai University of Illinois at Urbana-Champaign czhai@cs.uiuc.edu Abstract Probabilistic latent topic models have re- cently enjoyed much success in extracting and analyzing latent topics in text in an un- supervised way. One common deficiency of existing topic models, though, is that they would not work well for extracting cross-lingual latent topics simply because words in different languages generally do not co-occur with each other. In this paper, we propose a way to incorporate a bilin- gual dictionary into a probabilistic topic model so that we can apply topic models to extract shared latent topics in text data of different languages. Specifically, we pro- pose a new topic model called Probabilis- tic Cross-Lingual Latent Semantic Anal- ysis (PCLSA) which extends the Proba- bilistic Latent Semantic Analysis (PLSA) model by regularizing its likelihood func- tion with soft constraints defined based on a bilingual dictionary. Both qualitative and quantitative experimental results show that the PCLSA model can effectively extract cross-lingual latent topics from multilin- gual text data. 1 Introduction As a robust unsupervised way to perform shallow latent semantic analysis of topics in text, prob- abilistic topic models (Hofmann, 1999a; Blei et al., 2003b) have recently attracted much atten- tion. The common idea behind these models is the following. A topic is represented by a multino- mial word distribution so that words characteriz- ing a topic generally have higher probabilities than other words. We can then hypothesize the exis- tence of multiple topics in text and define a gener- ative model based on the hypothesized topics. By fitting the model to text data, we can obtain an es- timate of all the word distributions corresponding to the latent topics as well as the topic distributions in text. Intuitively, the learned word distributions capture clusters of words that co-occur with each other probabilistically. Although many topic models have been pro- posed and shown to be useful (see Section 2 for more detailed discussion of related work), most of them share a common deficiency: they are de- signed to work only for mono-lingual text data and would not work well for extracting cross-lingual latent topics, i.e. topics shared in text data in two different natural languages. The deficiency comes from the fact that all these models rely on co-occurrences of words forming a topical cluster, but words in different language generally do not co-occur with each other. Thus with the existing models, we can only extract topics from text in each language, but cannot extract common topics shared in multiple languages. In this paper, we propose a novel topic model, called Probabilistic Cross-Lingual Latent Seman- tic Analysis (PCLSA) model, which can be used to mine shared latent topics from unaligned text data in different languages. PCLSA extends the Proba- bilistic Latent Semantic …

Content truncated to 500 words