A Discriminative Topic Model using Document Network Structure

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 686–696, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics A Discriminative Topic Model using Document Network Structure Weiwei Yang Computer Science University of Maryland College Park, MD wwyang@cs.umd.edu Jordan Boyd-Graber Computer Science University of Colorado Boulder, CO Jordan.Boyd.Graber@ colorado.edu Philip Resnik Linguistics and UMIACS University of Maryland College Park, MD resnik@umd.edu Abstract Document collections often have links be- tween documents—citations, hyperlinks, or revisions—and which links are added is often based on topical similarity. To model these intuitions, we introduce a new topic model for documents situated within a net- work structure, integrating latent blocks of documents with a max-margin learning criterion for link prediction using topic- and word-level features. Experiments on a scientific paper dataset and collection of webpages show that, by more robustly exploiting the rich link structure within a document network, our model improves link prediction, topic quality, and block distributions. 1 Introduction Documents often appear within a network struc- ture: social media mentions, retweets, and fol- lower relationships; Web pages by hyperlinks; sci- entific papers by citations. Network structure in- teracts with the topics in the text, in that docu- ments linked in a network are more likely to have similar topic distributions. For instance, a cita- tion link between two papers suggests that they are about a similar field, and a mentioning link between two social media users often indicates common interests. Conversely, documents’ sim- ilar topic distributions can suggest links between them. For example, topic model (Blei et al., 2003, LDA) and block detection papers (Holland et al., 1983) are relevant to our research, so we cite them. Similarly, if a social media user A finds another user B with shared interests, then A is more likely to follow B. Our approach is part of a natural progression of network modeling in which models integrate more information in more sophisticated ways. Some past methods only consider the network it- self (Kim and Leskovec, 2012; Liben-Nowell and Kleinberg, 2007), which loses the rich information in text. In other cases, methods take both links and text into account (Chaturvedi et al., 2012), but they are modeled separately, not jointly, limiting the model’s ability to capture interactions between the two. The relational topic model (Chang and Blei, 2010, RTM) goes further, jointly modeling topics and links, but it considers only pairwise document relationships, failing to capture network structure at the level of groups or blocks of documents. We propose a new joint model that makes fuller use of the rich link structure within a document network. Specifically, our model embeds the weighted stochastic block model (Aicher et al., 2014, WSBM) to identify blocks in which docu- ments are densely connected. WSBM basically cat- egorizes each item in a network probabilistically as belonging to one of L blocks, by reviewing its connections with each block. Our model can be viewed as a principled probabilistic extension of Yang et al. (2015), who identify blocks in …

Content truncated to 500 words