Review Topic Discovery with Phrases using the Pólya Urn Model

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 667–676, Dublin, Ireland, August 23-29 2014. Review Topic Discovery with Phrases using the Pólya Urn Model Geli Fei Department of Computer Science, University of Illi- nois at Chicago, Chicago, USA gfei2@uic.edu Zhiyuan Chen Department of Computer Science, University of Illi- nois at Chicago, Chicago, USA czyuanacm@gmail.com Bing Liu Department of Computer Science, University of Illi- nois at Chicago, Chicago, USA liub@cs.uic.edu Abstract Topic modelling has been popularly used to discover latent topics from text documents. Most existing models work on individual words. That is, they treat each topic as a distribution over words. However, using only individual words has several shortcomings. First, it increases the co-occurrences of words which may be incorrect because a phrase with two words is not equivalent to two separate words. These extra and often incorrect co-occurrences result in poorer output topics. A multi-word phrase should be treated as one term by itself. Second, individual words are often difficult to use in practice because the meaning of a word in a phrase and the meaning of a word in isolation can be quite different. Third, topics as a list of individual words are also difficult to understand by users who are not domain experts and do not have any knowledge of topic models. In this paper, we aim to solve these problems by considering phrases in their natural form. One simple way to include phrases in topic modelling is to treat each phrase as a single term. However, this method is not ideal because the meaning of a phrase is often related to its composite words. That information is lost. This paper proposes to use the generalized Pólya Urn (GPU) model to solve the problem, which gives superior results. GPU enables the connection of a phrase with its content words naturally. Our experimental results using 32 review datasets show that the proposed approach is highly effective. 1 Introduction Topic models such as LDA (Blei et al., 2003) and pSLA (Hofmann 1999) and their extensions have been popularly used to find topics in text documents. These models are mostly governed by the phe- nomenon called “higher-order co-occurrence” (Heinrich 2009), i.e., how often terms co-occur in differ- ent contexts. Word w1 co-occurring with word w2 which in turn co-occurs with word w3 denotes a sec- ond-order co-occurrence between w1 and w3. Almost all these models regard each topic as a distribution over words. The words under each topic are often sorted according to their associated probabilities. Those top ranked words are used to represent the topic. However, this representation of topics as a list of individual words has some 1major shortcomings: • Topics are often difficult to understand or interpret by users unless they are domain experts and also knowledgeable about topic models. In most real-life situations, these are not the case. In some of our applications, we show users several good topics, but they have no idea what they are because many domain …

Content truncated to 500 words