Neural Topic Model with Reinforcement Learning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3478–3483, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3478 Neural Topic Model with Reinforcement Learning Lin Gui1,§, Jia Leng2,§, Gabriele Pergola1, Yu Zhou1, Ruifeng Xu2,3,4, Yulan He1,† 1Department of Computer Science, University of Warwick, UK 2Harbin Institute of Technology (Shenzhen), China 3Peng Cheng Laboratory, Shenzhen, China 4Joint Lab of Harbin Institute of Technology and RICOH Lin.gui@warwick.ac.uk, lengjia@stu.hit.edu.cn gabriele.pergola@warwick.ac.uk, Yu.Zhou.1@warwick.ac.uk xuruifeng@hit.edu.cn, yulan.he@warwick.ac.uk Abstract In recent years, advances in neural variational inference have achieved many successes in text processing. Examples include neural topic models which are typically built upon varia- tional autoencoder (VAE) with an objective of minimising the error of reconstructing original documents based on the learned latent topic vectors. However, minimising reconstruction errors does not necessarily lead to high quality topics. In this paper, we borrow the idea of re- inforcement learning and incorporate topic co- herence measures as reward signals to guide the learning of a VAE-based topic model. Fur- thermore, our proposed model is able to au- tomatically separating background words dy- namically from topic words, thus eliminating the pre-processing step of filtering infrequent and/or top frequent words, typically required for learning traditional topic models. Exper- imental results on the 20 Newsgroups and the NIPS datasets show superior performance both on perplexity and topic coherence measure compared to state-of-the-art neural topic mod- els. 1 Introduction Probabilistic topic models have been used widely in nature language processing (Li et al., 2016; Zeng et al., 2018). The fundamental principle is that words are assumed to be generated from la- tent topics which can be inferred from data based on word co-occurrence patterns (Neal, 1993; An- drieu et al., 2003). In recent years, Variational Autoencoder (VAE) has been proved more effec- tive and efficient to approximating deep, complex and underestimated variance in integrals (Kingma and Welling, 2013; He et al., 2017). However, the VAE-based topic models focus on the construc- tion of deep neural networks to approximate the §The two authors contributed equally to this work. †Corresponding author. intractable distribution between observed words and latent topics based on log-likelihood and the learning objective is to minimise the error of re- constructing the original documents based on the learned latent topic vectors rather than improving the quality of learned topics, for example, mea- sured by coherence scores (Kingma and Welling, 2013; Sønderby et al., 2016; Miao et al., 2016; Card et al., 2017; Srivastava and Sutton, 2017; Bouchacourt et al., 2018). The lack of considera- tion of topic coherence measures during the learn- ing process of VAE-based topic models makes it difficult to control the quality of the generated top- ics. Intuitively, one solution is to jointly consider coherence scores in the learning objective. How- ever, this is not feasible since coherence score is an unsupervised measure of topics based on a large- scale knowledge source, there is no ground truth “best topics”. Another limitation …

Content truncated to 500 words