CluHTM - Semantic Hierarchical Topic Modeling based on CluWords

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8138–8150 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 8138 CluHTM - Semantic Hierarchical Topic Modeling based on CluWords Felipe Viegas1, Washington Cunha1, Christian Gomes1, Antˆonio Pereira2, Leonardo Rocha2, Marcos Andr´e Gonc¸alves1 1 Universidade Federal de Minas Gerais - Brazil 2 Universidade Federal de S˜ao Jo˜ao del Rei - Brazil {frviegas, washingtoncunha}@dcc.ufmg.br {christianreis, mgoncalv}@dcc.ufmg.br {antoniopereira, lcrocha}@ufsj.edu.br Abstract Hierarchical Topic modeling (HTM) exploits latent topics and relationships among them as a powerful tool for data analysis and exploration. Despite advantages over tradi- tional topic modeling, HTM poses its own challenges, such as (1) topic incoherence, (2) unreasonable (hierarchical) structure, and (3) issues related to the definition of the “ideal” number of topics and depth of the hierarchy. In this paper, we advance the state- of-the-art on HTM by means of the design and evaluation of CluHTM, a novel non- probabilistic hierarchical matrix factorization aimed at solving the specific issues of HTM. CluHTM’s novel contributions include: (i) the exploration of richer text representation that encapsulates both, global (dataset level) and local semantic information – when combined, these pieces of information help to solve the topic incoherence problem as well as issues related to the unreasonable structure; (ii) the exploitation of a stability analysis metric for defining the number of topics and the “shape” the hierarchical structure. In our evaluation, considering twelve datasets and seven state- of-the-art baselines, CluHTM outperformed the baselines in the vast majority of the cases, with gains of around 500% over the strongest state-of-the-art baselines. We also provide qualitative and quantitative statistical analyses of why our solution works so well. 1 Introduction Topic Modeling (TM) is the task of automatically extracting latent topics (e.g., a concept or a theme) from a collection of textual documents. Such topics are usually defined as a probability distribution over a fixed vocabulary (a set of words) that refers to some subject and describes the latent topic as a whole. Topics might be related to each other, and if they are defined at different semantic granularity levels (more general or more specific), this nat- urally induces a hierarchical structure. Although traditional TM strategies are of great importance to extract latent topics, the relationships among them are also extremely valuable for data analysis and exploration. In this context, Hierarchical Topic Modeling (HTM) aims to achieve – to induce latent topics from text data while preserving the inherent hierarchical structure (Teh et al., 2006). Relevant scenarios have been shown to enjoy the usefulness of HTM, such as (i) hierarchical categorization of Web pages (Ming et al., 2010), (ii) extracting aspects hierarchies in reviews (Kim et al., 2013) and (iii) discovering research topics hierarchies in academic repositories (Paisley et al., 2014). Despite its practical importance and potential advantages over traditional TM, HTM poses its own challenges, the main ones being: (i) topic incoherence and (ii) unreasonable hierarchical structure. Topic Incoherence has to do with the need to learn meaningful topics. That is, …

Content truncated to 500 words