Mining bilingual topic hierarchies from unaligned text

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 992–1000, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Mining bilingual topic hierarchies from unaligned text Sumit Negi IBM Research India Vasant Kunj, Institutional Area New Delhi, India sumitneg@in.ibm.com Abstract Recent years have seen an exponential growth in the amount of multilingual text available on the web. This situation raises the need for novel applications for or- ganizing and accessing multilingual con- tent. Common examples of such applica- tions include Multilingual Topic Tracking, Cross-Language Information retrieval sys- tems etc. Most of these applications rely on the availability of multilingual lexical resources which require significant effort to create. In this paper we present an un- supervised method for building bilingual topic hierarchies. In a bilingual topic hi- erarchy, topics (where a topic is a distri- bution over words) are arranged in a hi- erarchical fashion with abstract topics ap- pearing near the root of the hierarchy and more concrete topics near the leaves. Such bilingual topic hierarchies can be useful for organizing bilingual corpus based on common topics, cross-lingual information retrieval and cross-lingual text classifica- tion. Our method builds upon the prior work done on Bayesian non-parametric in- ferencing of topic hierarchies and multi- lingual topic modeling to extract bilingual topic hierarchies from unaligned text. We demonstrate the effectiveness of our algo- rithm in extracting such topic hierarchies from a collection of bilingual text passages and FAQs. 1 Introduction The last few decades have seen an explosive growth in Internet accessibility in developing re- gions of the world. A significant part of this growth has been in countries that use languages other than English as their primary language (Chi- nese, Spanish, Arabic, Hindi etc). The increasing number of multilingual Internet users has resulted in a tremendous increase in the amount multilin- gual content that is available on the Web. This sit- uation raises the need for novel ways of organizing and accessing multilingual content. Work done on Cross Language Information Retrieval (CLIR) (Xu et al., 2001) i.e. retrieving information writ- ten in a language different from the language of the user’s query is one key attempt in this very di- rection. Similarly, there has been increased inter- est in multilingual text mining applications such as sentiment detection (Boiy and Moens, 2008), mining multilingual news feeds, cross lingual text categorization (Bel et al., 2003) etc. Most of these applications use bilingual dictionaries or lex- ical databases to perform such tasks. For instance, (Pouliquen et al., 2004) use the Eurovoc multilin- gual thesaurus for cross-lingual news topic track- ing. (Mihalcea et al., 2007) use a bilingual dic- tionary to generate subjectivity analysis resources for a given language. Similarly, CINDOR (Ruiz et al., 2000) uses interlingua resources to address the cross language information retrieval problem. Considering the importance of multilingual lexical resources efforts to (semi) automatically build/populate such resources from Web or other large text collections have gained prominence. For example, (Nagata et al., 2001) build a bilingual dictionary of …

Content truncated to 500 words