TopicNet: Making Additive Regularisation for Topic Modelling Accessible

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6745–6752 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6745 TopicNet: Making Additive Regularisation for Topic Modelling Accessible Victor Bulatov, Evgeny Egorov, Eugenia Veselova, Darya Polyudova, Vasiliy Alekseev, Alexey Goncharov, Konstantin Vorontsov Moscow Institute of Physics and Technology (National Research University) 9 Institutskiy per., Dolgoprudny, Moscow Region, 141701, Russian Federation viktor.bulatov@phystech.edu, egorov.eo@mipt.ru, veselova.er@phystech.edu, wasya.alekseev@gmail.com, darya.polyudova@phystech.edu, alex.goncharov@phystech.edu, k.v.vorontsov@phystech.edu Abstract This paper introduces TopicNet, a new Python module for topic modelling. This package, distributed under the MIT license, focuses on bringing additive regularization for topic modelling (ARTM) to non-specialists using a general-purpose high-level language. The module features include powerful model visualization techniques, various training strategies, semi-automated model selection, support for user-defined goal metrics, and a modular approach to topic model training. Keywords: topic modelling, Python, supervised learning, unsupervised learning, latent Dirichlet allocation, additive regulariza- tion for topic modelling, BigARTM 1. Introduction Topic modelling is a method to extract hidden word prob- ability distributions called topics from text corpora. Being primarily introduced to find latent topics in text documents, topic models have proven to be relevant in a wide range of contexts (Boyd-Graber et al., 2017). Over the time topic models have primarily been used for two purposes: either providing embeddings over collec- tions of text for various recommendation and search pur- poses or giving a cohesive clustering of data for purposes of data mining and extracting information about the collec- tion structure. The first problem requires the researcher to tune their models to perform well on labelled datasets, thus, reduc- ing the task to classification problem using obtained from topic model document embeddings as feature vectors. This makes topic models compete with various machine learning and deep learning algorithms in the field of this classical task. Topic modelling does not necessarily show the best results in terms of classification metrics, but it has an edge when it comes to interpretability. Each topic can be interpreted as a probability distribution over words in that topic, and each coordinate in a document embedding has a meaning as a probability for that document to contain a certain topic in it. This property of topic models makes them better suited for the fields of AI where clarity of prediction matters or where one needs to correct undesirable biases introduced by the data into the model. It should be noted that topic modelling is still a valuable tool in natural language processing (NLP) researcher arse- nal. Currently, there is an ongoing work to extend state- of-the-art deep learning models to incorporate topical im- portance of words for text summarization (Narayan et al., 2018; Wang et al., 2018; Lebanoff et al., 2019) and song lyrics understanding (Choi et al., 2015; Fell et al., 2019; Watanabe et al., 2018). One can see that topic modelling and its numerous variations continues to be a workhorse in many NLP projects. The second purpose — collection structure retrieval, how- ever, is somehow unique to …

Content truncated to 500 words