Auto-Encoding Variational Bayes for Inferring Topics and Visualization

Proceedings of the 28th International Conference on Computational Linguistics, pages 5223–5234 Barcelona, Spain (Online), December 8-13, 2020 5223 Auto-Encoding Variational Bayes for Inferring Topics and Visualization Dang Pham, Tuan M. V. Le Department of Computer Science New Mexico State University {dangpnh, tuanle}@nmsu.edu Abstract Visualization and topic modeling are widely used approaches for text analysis. Traditional visu- alization methods find low-dimensional representations of documents in the visualization space (typically 2D or 3D) that can be displayed using a scatterplot. In contrast, topic modeling aims to discover topics from text, but for visualization, one needs to perform a post-hoc embedding using dimensionality reduction methods. Recent approaches propose using a generative model to jointly find topics and visualization, allowing the semantics to be infused in the visualization space for a meaningful interpretation. A major challenge that prevents these methods from being used practically is the scalability of their inference algorithms. We present, to the best of our knowledge, the first fast Auto-Encoding Variational Bayes based inference method for jointly inferring topics and visualization. Since our method is black box, it can handle model changes efficiently with little mathematical rederivation effort. We demonstrate the efficiency and effec- tiveness of our method on real-world large datasets and compare it with existing baselines. 1 Introduction Visualization and topic modeling are important tools in the analysis of text corpora. Visualization meth- ods, such as t-SNE (Maaten and Hinton, 2008), find low-dimensional representations of documents in the visualization space (typically 2D or 3D) that can be displayed using a scatterplot. Such visualization is useful for exploratory tasks. However, there is a lack of semantic interpretation as those visualization methods do not extract topics. In contrast, topic modeling aims to discover semantic topics from text, but for visualization, one needs to perform a post-hoc embedding using dimensionality reduction methods. Since this pipeline approach may not be ideal, there has been recent interest in jointly inferring topics and visualization using a single generative model (Iwata et al., 2008). This joint approach allows the semantics to be infused in the visualization space where users can view documents and their topics. The problem of jointly inferring topics and visualization can be formally stated as follows. Problem. Let D = {wn}N n=1 denote a finite set of N documents and let V be a finite vocabulary from these documents. Given a number of topics Z, and visualization dimension d, we want to find: • For topic modeling: Z latent topics, and their word distributions collectively denoted as β = {βz}Z z=1, topic distributions of documents collectively denoted as Θ = {θn}N n=1, and • For visualization: d-dimensional visualization coordinates for N documents X = {xn}N n=1, and Z topics Φ = {φz}Z z=1 such that the distances between documents, topics in the visualization space reflect the topic-document distributions Θ. To solve this problem, PLSV (Probabilistic Latent Semantic Visualization) is the first model that at- tempts to tie together all latent variables of topics and visualization (i.e., Υ = {X, Φ, β}) in a …

Content truncated to 500 words