A Disentangled Adversarial Neural Topic Model for Separating Opinions from Plots in User Reviews

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2870–2883 June 6–11, 2021. ©2021 Association for Computational Linguistics 2870 A Disentangled Adversarial Neural Topic Model for Separating Opinions from Plots in User Reviews Gabriele Pergola, Lin Gui, Yulan He Department of Computer Science, University of Warwick, UK {gabriele.pergola, lin.gui, yulan.he}@warwick.ac.uk Abstract The flexibility of the inference process in Vari- ational Autoencoders (VAEs) has recently led to revising traditional probabilistic topic mod- els giving rise to Neural Topic Models (NTM). Although these approaches have achieved sig- nificant results, surprisingly very little work has been done on how to disentangle the la- tent topics. Existing topic models when ap- plied to reviews may extract topics associated with writers’ subjective opinions mixed with those related to factual descriptions such as plot summaries in movie and book reviews. It is thus desirable to automatically separate opinion topics from plot/neutral ones enabling a better interpretability. In this paper, we pro- pose a neural topic model combined with ad- versarial training to disentangle opinion top- ics from plot and neutral ones. We conduct an extensive experimental assessment intro- ducing a new collection of movie and book re- views paired with their plots, namely MOBO dataset, showing an improved coherence and variety of topics, a consistent disentanglement rate, and sentiment classification performance superior to other supervised topic models. 1 Introduction Variational Autoencoders (VAEs) (Kingma et al., 2014) allow to design complex generative models of data. In the wake of the renewed interest for VAEs, traditional probabilistic topic models (Blei et al., 2003) have been revised giving rise to sev- eral Neural Topic Model (NTM) variants, such as NVDM (Miao et al., 2016), ProdLDA (Srivastava et al., 2017), and NTM-R (Ding et al., 2018). Al- though these approaches have achieved significant results via the neural inference process, existing topic models when applied to user reviews may ex- tract topics with writers’ subjective opinions mixed with those related to factual descriptions such as plot summaries of movies and books (Lin et al., 2012). Yet surprisingly very little work has been Gandalf Frodo Jackson Tolkien Dwarf Fellowship . . . Watch Sure Hobbit Trilogy Download Purchase . . . Silly Dwarf Lotr Jackson Shame Remove . . . . . . . . . “The Hobbit” reviews Disentangled Topics Positive Plot Negative Figure 1: Disentangled topics extracted by DIATOM from the Amazon reviews for “The Hobbit”. done on how to disentangle the inferred topic rep- resentations. Disentangled representations can be defined as representations where individual latent units are sensitive to variations of a single generative factor, while being relatively invariant to changes of other factors (Bengio et al., 2013; Higgins et al., 2017). Inducing such representations has been shown sig- nificantly beneficial for their generalization and interpretability (Achille and Soatto, 2018; Peng et al., 2019). In the context of topic modeling, we propose to consider latent topics as generative factors to be disentangled to improve their inter- pretability and …

Content truncated to 500 words