Fast, Flexible Models for Discovering Topic Correlation across Weakly-Related Collections

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1554–1564, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Fast, Flexible Models for Discovering Topic Correlation across Weakly-Related Collections Jingwei Zhang1, Aaron Gerow2, Jaan Altosaar3, James Evans2,4, Richard Jean So5 1Department of Computer Science, Columbia University jz2541@columbia.edu 2 Computation Institute, University of Chicago {gerow,jevans}@uchicago.edu 3 Department of Physics, Princeton University altosaar@princeton.edu 4 Department of Sociology, University of Chicago 5 Department of English Language and Literature, University of Chicago richardjeanso@uchicago.edu Abstract Weak topic correlation across document collections with different numbers of topics in individual collections presents challenges for existing cross-collection topic models. This paper introduces two probabilistic topic models, Correlated LDA (C-LDA) and Correlated HDP (C- HDP). These address problems that can arise when analyzing large, asymmetric, and potentially weakly-related collections. Topic correlations in weakly-related col- lections typically lie in the tail of the topic distribution, where they would be over- looked by models unable to fit large num- bers of topics. To efficiently model this long tail for large-scale analysis, our mod- els implement a parallel sampling algo- rithm based on the Metropolis-Hastings and alias methods (Yuan et al., 2015). The models are first evaluated on syn- thetic data, generated to simulate vari- ous collection-level asymmetries. We then present a case study of modeling over 300k documents in collections of sciences and humanities research from JSTOR. 1 Introduction Comparing large text collections is a critical task for the curation and analysis of human cultural history. Achievements of research and schol- arship are most accessible through textual arti- facts, which are increasingly available in digital archives. Text-based research, often undertaken by humanists, historians, lexicographers, and cor- pus linguists, explores patterns of words in docu- ments across time-periods and distinct collections of text. Here, we introduce two new topic models designed to compare large collections, Correlated LDA (C-LDA) and Correlated HDP (C-HDP), which are sensitive to document-topic asymme- try (where collections have different topic distribu- tions) and topic-word asymmetry (where a single topic has different word distributions in each col- lection). These models seek to address termino- logical questions, such as how a topic on physics is articulated distinctively in scientific compared to humanistic research. Accommodating poten- tial collection-level asymmetries is particularly important when researchers seek to analyze col- lections with little prior knowledge about shared or collection-specific topic structure. Our mod- els extend existing cross-collection approaches to accommodate these asymmetries and implement an efficient parallel sampling algorithm enabling users to examine the long tail of topics in particu- larly large collections. Using topic models for comparative text min- ing was introduced by Zhai et al. (2004), who de- veloped the ccMix model which extended pLSA (Hofmann, 1999). Later work by Paul and Girju (2009) developed ccLDA, which adopted the hier- archical Bayes framework of Latent Dirichlet Al- location or LDA (Blei et al., 2003). These mod- els account for topic-word asymmetry by assum- ing variation in the vocabularies of topics is due to collection-level differences. Nevertheless, …

Content truncated to 500 words