SSHLDA: A Semi-Supervised Hierarchical Topic Model

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 800–809, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics SSHLDA: A Semi-Supervised Hierarchical Topic Model Xian-Ling Mao♠∗, Zhao-Yan Ming♥, Tat-Seng Chua♥, Si Li♣, Hongfei Yan♠†, Xiaoming Li♠ ♠Department of Computer Science and Technology, Peking University, China ♥School of Computing, National University of Singapore, Singapore ♣School of ICE, Beijing University of Posts and Telecommunications, China {xianlingmao,lxm}@pku.edu.cn, yhf@net.pku.edu.cn {chuats,mingzhaoyan}@nus.edu.sg, lisi@bupt.edu.cn Abstract Supervised hierarchical topic modeling and unsupervised hierarchical topic modeling are usually used to obtain hierarchical topics, such as hLLDA and hLDA. Supervised hierarchi- cal topic modeling makes heavy use of the in- formation from observed hierarchical labels, but cannot explore new topics; while unsu- pervised hierarchical topic modeling is able to detect automatically new topics in the data space, but does not make use of any informa- tion from hierarchical labels. In this paper, we propose a semi-supervised hierarchical topic model which aims to explore new topics auto- matically in the data space while incorporating the information from observed hierarchical la- bels into the modeling process, called Semi- Supervised Hierarchical Latent Dirichlet Al- location (SSHLDA). We also prove that hLDA and hLLDA are special cases of SSHLDA. We conduct experiments on Yahoo! Answers and ODP datasets, and assess the performance in terms of perplexity and clustering. The ex- perimental results show that predictive ability of SSHLDA is better than that of baselines, and SSHLDA can also achieve significant im- provement over baselines for clustering on the FScore measure. 1 Introduction Topic models, such as latent Dirichlet allocation (LDA), are useful NLP tools for the statistical anal- ysis of document collections and other discrete data. ∗This work was done in National University of Singapore. †Corresponding author. Furthermore, hierarchical topic modeling is able to obtain the relations between topics — parent-child and sibling relations. Unsupervised hierarchical topic modeling is able to detect automatically new topics in the data space, such as hierarchical La- tent Dirichlet Allocation (hLDA) (Blei et al., 2004). hLDA makes use of nested Dirichlet Process to auto- matically obtain a L-level hierarchy of topics. Mod- ern Web documents, however, are not merely col- lections of words. They are usually documents with hierarchical labels – such as Web pages and their placement in hierarchical directories (Ming et al., 2010). Unsupervised hierarchical topic modeling cannot make use of any information from hierarchi- cal labels, thus supervised hierarchical topic models, such as hierarchical Labeled Latent Dirichlet Allo- cation (hLLDA) (Petinot et al., 2011), are proposed to tackle this problem. hLLDA uses hierarchical la- bels to automatically build corresponding topic for each label, but it cannot find new latent topics in the data space, only depending on hierarchy of labels. As we know that only about 10% of an iceberg’s mass is seen outside while about 90% of it is unseen, deep down in water. We think that a corpus with hi- erarchical labels should include not only observed topics of labels, …

Content truncated to 500 words