A Bayesian Topic Model for Human-Evaluated Interpretability

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6271–6279 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6271 A Bayesian Topic Model for Human-Evaluated Interpretability Justin Wood, Corey Arnold, Wei Wang University of California, Los Angeles {juwood03, cwarnold, weiwang}@ucla.edu Abstract One desideratum of topic modeling is to produce interpretable topics. Given a cluster of document-tokens comprising a topic, we can order the topic by counting each word. It is natural to think that each topic could easily be labeled by looking at the words with the highest word count. However, this is not always the case. A human evaluator can often have difficulty identifying a single label that accurately describes the topic as many top words seem unrelated. This paper aims to improve interpretability in topic modeling by providing a novel, outperforming interpretable topic model. Our approach combines two previously established subdomains in topic modeling: nonparametric and weakly-supervised topic models. Given a nonparametric topic model, we can include weakly-supervised input using novel modifications to the nonparametric generative model. These modifications lay the groundwork for a compelling setting—one in which most corpora, without any previous supervised or weakly-supervised input, can discover interpretable topics. This setting also presents various challenging sub-problems of which we provide resolutions. Combining nonparametric topic models with weakly-supervised topic models leads to an exciting discovery—a complete, self-contained and outperforming topic model for interpretability. Keywords: Topic modeling, Weak supervision, Topic interpretability 1. Introduction Topic modeling is an effective way to analyze un- structured textual data. Although recent topic mod- eling research is often performed using deep neural networks (Duan and others, 2021; Chen and others, 2021; Rezaee and others, 2020), an alternative technique for topic discovery is based on a Bayesian graphical model (Blei and others, 2003b). The basic assumption of these Bayesian models consists of a generative model for the input text. Words are generated by first sampling a topic assignment from a document-level topic distri- bution (θ). Then for the topic assignment, a word is generated from the corresponding topic-level word dis- tribution (φ). This process is completed over the entire length of the corpus. Inference of the two hidden dis- tributions is made using Bayesian inference techniques such as Gibbs sampling (Griffiths and Steyvers, 2004; Griffiths, 2002). The topics themselves consist of word assignments from the corpus to each topic. These word assignments are clustered together to form distributions. Since a topic is only a word assignment distribution, it is not always the case that a single n-gram can describe the topic. A topic without a single n-gram description represents a diver- gence from how a layperson might think of what a topic is—which could be: the subject of a discourse or of a section of a discourse (Topic., 2021). This divergence is at the center of interpretability. Interpretable topics bridge this gap by providing the cluster of words that can be described with a single label that best explains the topic1. For example, if the …

Content truncated to 500 words