TSDPMM: Incorporating Prior Topic Knowledge into Dirichlet Process Mixture Models for Text Clustering

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 787–792, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. TSDPMM: Incorporating Prior Topic Knowledge into Dirichlet Process Mixture Models for Text Clustering Linmei Hu†, Juanzi Li†, Xiaoli Li‡, Chao Shao†, Xuzhong Wang§ † Dept. of Computer Sci. and Tech., Tsinghua University, China ‡ Institute for Infocomm Research(I2R), A*STAR, Singapore § State Key Laboratory of Math. Eng. and Advanced Computing, China {hulinmei1991, lijuanzi2008}@gmail.com xlli@i2r.a-star.edu.sg, {birdlinux, koodoneko}@gmail.com Abstract Dirichlet process mixture model (DPM- M) has great potential for detecting the underlying structure of data. Extensive studies have applied it for text cluster- ing in terms of topics. However, due to the unsupervised nature, the topic cluster- s are always less satisfactory. Considering that people often have some prior knowl- edge about which potential topics should exist in given data, we aim to incorpo- rate such knowledge into the DPMM to improve text clustering. We propose a novel model TSDPMM based on a new seeded P´olya urn scheme. Experimen- tal results on document clustering across three datasets demonstrate our proposed TSDPMM significantly outperforms state- of-the-art DPMM model and can be ap- plied in a lifelong learning framework. 1 Introduction Dirichlet process mixture model (DPMM) (Neal, 2000) has been used in detecting the underlying structure in data. For example, (Vlachos et al., 2008; Vlachos et al., 2009) applied it to lexical- semantic verb clustering. (Wang et al., 2011; Huang et al., 2013; Yin and Wang, 2014) applied it for text clustering in terms of their topics. While DPMM achieved some promising results, it can still sometimes produce unsatisfactory topic clus- ters due to its unsupervised nature. On the other hand, people often have prior knowledge about what potential topics should ex- ist in a given text corpus. Take an earthquake event corpus as an example. The topics, such as “ca- sualties and damages”, “rescue” and “government reaction”, called prior topics, are expected to oc- cur in the corpus according to our common knowl- edge (e.g., the topics automatically learned from previous events using topic modeling (Ahmed and Xing, 2008)) or external resources (e.g., table of contents at Wikipedia event pages 1). Similarly, in academic fields, “call for papers (CFP)” of confer- ences 2 lists main topics that conference organizers would like to focus on. Clearly, these prior topic- s can be represented as sets of words, which are available in many real-world applications. They can serve as weakly supervised information to en- hance the unsupervised DPMM for text clustering. Standard DPMM (Neal, 2000; Ranganathan, 2006) lacks a mechanism for incorporating pri- or knowledge. Some existing work (Vlachos et al., 2008; Vlachos et al., 2009) added knowledge of observed instance-level constraints (must-links and cannot-links between documents) to DPMM. (Ahmed and Xing, 2008) proposed recurrent Chi- nese Restaurant Process to incorporate previous documents with known topic clusters. We focus on incorporating topic-level knowledge, which is more challenging, as seed/prior topics could be la- tent rather than observable. Particularly, we construct our …

Content truncated to 500 words