HTMOT: Hierarchical Topic Modelling over Time

Proceedings of Recent Advances in Natural Language Processing, pages 854–863 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_092 854 HTMOT : Hierarchical Topic Modelling Over Time Judicael Poumay ULiege/HEC Liege Rue Louvrex 14, 4000 Liege, Belgium judicael.poumay@uliege.be Ashwin Ittoo ULiege/HEC Liege Rue louvrex 14, 4000 Liege, Belgium ashwin.ittoo@uliege.be Abstract Topic models provide an efficient way of extracting insights from text and supporting decision-making. Recently, novel methods have been proposed to model topic hierarchy or temporality. Modeling temporality provides more precise topics by separating topics that are characterized by similar words but located over distinct time periods. Conversely, mod- eling hierarchy provides a more detailed view of the content of a corpus by providing topics and sub-topics. However, no models have been proposed to incorporate both hierarchy and tem- porality which could be beneficial for applica- tions such as environment scanning. Therefore, we propose a novel method to perform Hierar- chical Topic Modelling Over Time (HTMOT). We evaluate the performance of our approach on a corpus of news articles using the Word Intrusion task. Results demonstrate that our model produces topics that elegantly combine a hierarchical structure and a temporal aspect. Furthermore, our proposed Gibbs sampling im- plementation shows competitive performance compared to previous state-of-the-art methods. 1 Introduction In the field of natural language processing (NLP), numerous methods for extracting topics from a corpus have been proposed over the years (Al- ghamdi and Alfalqi, 2015; Barde and Bainwad, 2017). While the seminal Latent Dirichlet Allo- cation (LDA) algorithm (Blei et al., 2003) paved the way for topic modeling, it lacks the ability to capture hierarchical or temporal information. In the past, hierarchical topic models have been proposed (Paisley et al., 2015; Blei et al., 2004) that enable the extraction of topics and sub-topics organized in a tree-like structure. These models dynamically determine the appropriate number of topics and sub-topics during training and have been found to be useful in ontology learning (Zhu et al., 2017) and research idea recommendation (Wang et al., 2019). In parallel, temporal topic models have been developed (Wang and McCallum, 2006; Nallapati et al., 2007; Song et al., 2008; Blei and Lafferty, 2006) that allow for the extraction of topics that describe events or trends occurring in a corpus. These models have been applied to tasks such as tracking trends in scientific articles (Hong et al., 2011) and events in social media (Zhou and Chen, 2013). Combining hierarchical and temporal informa- tion in models can capture broad and detailed as- pects of a corpus, benefiting applications like envi- ronment scanning (El Akrouchi et al., 2021). Hi- erarchical modeling yields detailed topics and sub- topics for a comprehensive thematic understanding, while temporal modeling provides precise descrip- tions of events. This integration produces nuanced models for informed decision-making and deeper insights. However, integrating temporal and hierarchical information in topic models remains a challenge (Nallapati et al., 2007; Song et al., 2008; Blei and Lafferty, 2006; Wang and McCallum, 2006). Many temporal models have their own structures to represent time, such as time …

Content truncated to 500 words