A Semantic Cover Approach for Topic Modeling

Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM), pages 92–102 Minneapolis, June 6–7, 2019. c⃝2019 Association for Computational Linguistics 92 A Semantic Cover Approach for Topic Modeling Rajagopal Venkatesaramani Computer Science, WUSTL St. Louis MO 63130 rajagopal@wustl.edu Douglas Downey EECS, Northwestern University Evanston IL 60208 ddowney@eecs.northwestern.edu Bradley Malin Biomedical Informatics, Vanderbilt University Nashville TN 37209 b.malin@vanderbilt.edu Yevgeniy Vorobeychik Computer Science, WUSTL St. Louis MO 63130 yvorobeychik@wustl.edu Abstract We introduce a novel topic modeling approach based on constructing a semantic set cover for clusters of similar documents. Specifi- cally, our approach first clusters documents us- ing their Tf-Idf representation, and then covers each cluster with a set of topic words based on semantic similarity, defined in terms of a word embedding. Computing a topic cover amounts to solving a minimum set cover prob- lem. Our evaluation compares our topic mod- eling approach to Latent Dirichlet Allocation (LDA) on three metrics: 1) qualitative topic match, measured using evaluations by Ama- zon Mechanical Turk (MTurk) workers, 2) per- formance on classification tasks using each topic model as a sparse feature representation, and 3) topic coherence. We find that qual- itative judgments significantly favor our ap- proach, the method outperforms LDA on topic coherence, and is comparable to LDA on doc- ument classification tasks. 1 Introduction Topic modeling is one of the core research prob- lems in natural language processing. Approaches to topic modeling range from simple vector com- parisons to probabilistic graphical models (Deer- wester et al., 1990; Hofmann, 1999; Blei et al., 2003; Mimno and McCallum, 2012). Neverthe- less, despite the many approaches proposed over the years, probabilistic topic modeling methods in general, and Latent Dirichlet Allocation (LDA) (Blei et al., 2003) in particular, have become ar- guably the dominant paradigm. For example, it remains the algorithm of choice in the Amazon’s healthcare NLP toolkit (Amazon Web Services, 2018). However, there have been concerns about the performance of probabilistic models, particularly in the context of datasets comprised of short doc- uments, such as tweets (Davidson et al., 2017; Yan et al., 2013; Hong and Davison, 2010; Mit- tos et al., 2018; Steinskog et al., 2017). This is primarily because the sparsity posed by short texts makes it hard for the model to sufficiently account for word co-occurrences, which form the basis of the definition of a topic in the sense of a multi- nomial distribution over words. Additionally, the language used on Twitter is informal in nature, uses slang and non-dictionary words, and often lacks proper grammatical structure. Moreover, the complexity of the probabilistic topic modeling ap- proaches makes it difficult to interpret the spe- cific choices they make about topics and their con- stituent words. In this paper, we propose a novel approach to topic modeling which is conceptually simple and highly interpretable. Our approach is based on two hypotheses about the nature of short texts, such as tweets: first, that such texts can be grouped into relatively few disjoint clusters representing a similar mix of subjects …

Content truncated to 500 words