Birds of a Feather Linked Together: A Discriminative Topic Model using Link-based Priors

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 261–266, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Birds of a Feather Linked Together: A Discriminative Topic Model using Link-based Priors Weiwei Yang Computer Science University of Maryland College Park, MD wwyang@cs.umd.edu Jordan Boyd-Graber Computer Science University of Colorado Boulder, CO Jordan.Boyd.Graber@ colorado.edu Philip Resnik Linguistics and UMIACS University of Maryland College Park, MD resnik@umd.edu Abstract A wide range of applications, from social media to scientific literature analysis, in- volve graphs in which documents are con- nected by links. We introduce a topic model for link prediction based on the in- tuition that linked documents will tend to have similar topic distributions, integrat- ing a max-margin learning criterion and lexical term weights in the loss function. We validate our approach on the tweets from 2,000 Sina Weibo users and evalu- ate our model’s reconstruction of the so- cial network. 1 Introduction Many application areas for text analysis involve documents connected by links of one or more types—for example, analysis of scientific pa- pers (citations, co-authorship), Web pages (hyper- links), legislation (co-sponsorship, citations), and social media (followers, mentions, etc.). In this paper we work within the widely used framework of topic modeling (Blei et al., 2003, LDA) to de- velop a model that is simple and intuitive, but which identifies high quality topics while also ac- curately predicting link structure. Our work here is inspired by the phenomenon of homophily, the tendency of people to associate with others who are like themselves (McPherson et al., 2001). As manifested in social networks, the intuition is that people who are associated with one another are likely to discuss similar topics, and vice versa. The new topic model we propose there- fore takes association links into account so that a document’s topic distribution is influenced by the topic distributions of its neighbors. Specifically, we propose a joint model that uses link structure to define clusters (cliques) of documents and, fol- lowing the intuition that documents in the same cluster are likely to have similar topic distribu- tions, assigns each cluster its own separate Dirich- let prior over the cluster’s topic distribution. This use of priors is consistent with previous work that has shown document-topic priors to be useful in encoding various types of prior knowledge and improving topic modeling performance (Mimno and McCallum, 2008). We then use distributed representations to “seed” the topic representations before getting down to modeling the documents. Our joint objective function uses a discriminative, max-margin approach (Zhu et al., 2012; Zhu et al., 2014) to both model the contents of documents and produce good predictions of links; in addition, it improves prediction by including lexical terms in the decision function (Nguyen et al., 2013). Our baseline for comparison is the Relational Topic Model (Chang and Blei, 2010, henceforth RTM), which jointly captures topics and binary link indicators in a style similar to supervised LDA (McAuliffe and Blei, 2008, sLDA), instead of modeling links alone, …

Content truncated to 500 words