Benchmarking Neural Topic Models: An Empirical Study

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4363–4368 August 1–6, 2021. ©2021 Association for Computational Linguistics 4363 Benchmarking Neural Topic Models: An Empirical Study Thanh-Nam Doan University of Tennessee at Chattanooga Chattanooga, TN, USA thanh-nam-doan@utc.edu Tuan-Anh Hoang VNU University of Science 334 Nguyen Trai, Hanoi, Vietnam hoangtuananh@hus.edu.vn Abstract Neural topic modeling approach has been at- tracting much attention recently as it is able to leverage the advantages of both neural net- works and probabilistic topic models. Previ- ous works have proposed several models that are based on this framework and obtained im- pressive experimental results compared to tra- ditional probabilistic models. However, the re- ported result is not consistent across the works, making them hard for gaining a rigorous as- sessment of these approaches. This work aims to address this issue by offering an extensive empirical evaluation of typical neural topic models in different aspects using large, di- verse datasets as well as a thorough set of met- rics. Precisely, we examine the performance of these models in three tasks, namely uncov- ering cohesive topics, modeling the input doc- uments, and representing them for downstream classification. Our results show that while the neural topic models are better in the first and the third tasks, the traditional probabilis- tic models are still a strong baseline and are better in the second task in many cases. These findings give us more insights for choosing off- the-shelf topic modeling toolboxes in different contexts, as well as for designing more com- prehensive evaluation for neural topic models. 1 Introduction Classical topic modeling approach consists of sta- tistical learning methods for uncovering the latent topics from a corpus and the semantic meaning of each document in the corpus. Notable works include the pioneering ones by (Hofmann, 1999; Blei et al., 2003). During its more than 20 years of research, topic modeling has also been applied to other fields beyond its original scope such as image analysis (Fei-Fei and Perona, 2005), and recom- mender systems (McAuley and Leskovec, 2013). Recently, neural topic modeling approach has been attracting much research attention for under- standing topic models of corpus due to its ability to leverage the advantages of both neural networks and probabilistic generative models. Compared to classical ones, this approach has three main advan- tages. First, its inference is amortized (Miao et al., 2016) and hence is much computationally simpler than that of the classical approach which requires to deal with a complicated optimization problem (Blei et al., 2003). Second, the gap between prototype and deployment processes become closer thanks to the power of some deep learning frameworks such as Pytorch (Paszke et al., 2019), Tensorflow (Abadi et al., 2016), and Flux.jl (Innes, 2018). Third, neu- ral topic models are easy to be integrated with prior knowledge such as pre-trained word and text em- beddings, which are prevalent and have shown the tremendous usefulness, e.g. GPT-3 (Brown et al., 2020), BERT (Devlin et al., 2019). Several neural topic models are proposed re- cently to …

Content truncated to 500 words