HTM: A Topic Model for Hypertexts

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 514–522, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics HTM: A Topic Model for Hypertexts Congkai Sun∗ Department of Computer Science Shanghai Jiaotong University Shanghai, P. R. China martinsck@hotmail.com Bin Gao Microsoft Research Asia No.49 Zhichun Road Beijing, P. R. China bingao@microsoft.com Zhenfu Cao Department of Computer Science Shanghai Jiaotong University Shanghai, P. R. China zfcao@cs.sjtu.edu.cn Hang Li Microsoft Research Asia No.49 Zhichun Road Beijing, P. R. China hangli@microsoft.com Abstract Previously topic models such as PLSI (Prob- abilistic Latent Semantic Indexing) and LDA (Latent Dirichlet Allocation) were developed for modeling the contents of plain texts. Re- cently, topic models for processing hyper- texts such as web pages were also proposed. The proposed hypertext models are generative models giving rise to both words and hyper- links. This paper points out that to better rep- resent the contents of hypertexts it is more es- sential to assume that the hyperlinks are fixed and to define the topic model as that of gen- erating words only. The paper then proposes a new topic model for hypertext processing, referred to as Hypertext Topic Model (HTM). HTM defines the distribution of words in a document (i.e., the content of the document) as a mixture over latent topics in the document itself and latent topics in the documents which the document cites. The topics are further characterized as distributions of words, as in the conventional topic models. This paper fur- ther proposes a method for learning the HTM model. Experimental results show that HTM outperforms the baselines on topic discovery and document classification in three datasets. 1 Introduction Topic models are probabilistic and generative mod- els representing contents of documents. Examples of topic models include PLSI (Hofmann, 1999) and LDA (Blei et al., 2003). The key idea in topic mod- eling is to represent topics as distributions of words * This work was conducted when the first author visited Microsoft Research Asia as an intern. and define the distribution of words in document (i.e., the content of document) as a mixture over hid- den topics. Topic modeling technologies have been applied to natural language processing, text min- ing, and information retrieval, and their effective- ness have been verified. In this paper, we study the problem of topic mod- eling for hypertexts. There is no doubt that this is an important research issue, given the fact that more and more documents are available as hypertexts cur- rently (such as web pages). Traditional work mainly focused on development of topic models for plain texts. It is only recently several topic models for pro- cessing hypertexts were proposed, including Link- LDA and Link-PLSA-LDA (Cohn and Hofmann, 2001; Erosheva et al., 2004; Nallapati and Cohen, 2008). We point out that existing models for hypertexts may not be suitable for characterizing contents of hypertext documents. This is because all the models are assumed to generate both words and hyperlinks (outlinks) of documents. The generation of the latter type …

Content truncated to 500 words