Exploring Topic Discriminating Power of Words in Latent Dirichlet Allocation

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2238–2247, Osaka, Japan, December 11-17 2016. Exploring Topic Discriminating Power of Words in Latent Dirichlet Allocation Kai Yang, Yi Cai∗, Zhenhong Chen School Of Software Engineering, South China University of Technology, China Ho-fung Leung Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong Raymond LAU College of Business, City University of Hong Kong, Hong Kong Abstract Latent Dirichlet Allocation (LDA) and its variants have been widely used to discover latent topics in textual documents. However, some of topics generated by LDA may be noisy with irrelevant words scattering across these topics. We name this kind of words as topic-indiscriminate words, which tend to make topics more ambiguous and less interpretable by humans. In our work, we propose a new topic model named TWLDA, which assigns low weights to words with low topic discriminating power (ability). Our experimental results show that the proposed approach, which effectively reduces the number of topic-indiscriminate words in discovered topics, improves the effectiveness of LDA. 1 Introduction Latent Dirichlet Allocation (LDA) (Blei et al., 2003) and its variants are generative statistical topic models providing a powerful framework for finding topics in text documents. In generative process, each document is a mixture of several topics, and the generation of each word belongs to one of the document’s topics (Heinrich, 2009). Mimno et al. have found that LDA often produces topics that are not interpretable or meaningful (Mimno et al., 2011). According to our observation, most of topics (especially those considered uninterpretable) contain some words which are common in the corpus. For example, words like ‘science’, ‘academic’ or ‘abstract’ in a corpus about scientific publications will appear in most of topics. To explain this kind of words more clearly, we prepare another example showed in Table 1(a). The table shows the top 5 words for 5 topics generated by standard LDA from a corpus of reviews about smart phones. Word ‘phone’ can be easily recognized as a common word in the corpus about phones, and we can find that all the topics contain this word. For words which are likely to scatter across many topics are difficult to discriminate different topics, we denote this kind of words, such as ‘phone’, as topic-indiscriminate words. We use the term topic discriminating power to denote the ability of a word discriminating different topics. Topic-indiscriminate words have low topic discriminating power. Table 1: An example about a result of standard LDA (a) Result of standard LDA Topic Word 1 sound, headphones, phone, bass, card 2 screen, iphone, phone, display, ear 3 picture, phone, photo, video, gb 4 memory, sd, gb, phone, battery 5 android, phone, nexus, Samsung, google (b) Document frequency (DF) of words Word DF Word DF phone 2041 screen 1900 memory 1553 picture 1451 sound 1221 sd 928 android 915 iphone 837 headphones 428 card 389 1Corresponding author, e-mail: ycai@scut.edu.cn This work is licensed under a Creative Commons Attribution 4.0 International …

Content truncated to 500 words