Combining Sentiment Lexica with a Multi-View Variational Autoencoder

Proceedings of NAACL-HLT 2019, pages 635–640 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 635 Combining Sentiment Lexica with a Multi-View Variational Autoencoder Alexander Hoyle@ Lawrence Wolf-SonkinS Hanna WallachZ Ryan CotterellH Isabelle AugensteinP @University College London, London, UK SDepartment of Computer Science, Johns Hopkins University, Baltimore, USA ZMicrosoft Research, New York City, USA HDepartment of Computer Science and Technology, University of Cambridge, Cambridge, UK PDepartment of Computer Science, University of Copenhagen, Copenhagen, Denmark alexander.hoyle.17@ucl.ac.uk, lawrencews@jhu.edu hanna@dirichlet.net, rdc42@cam.ac.uk, augenstein@di.ku.dk Abstract When assigning quantitative labels to dataset, different methodologies may rely ona different scales. In particular, when assigning polarities to words in a sentiment lexicon, annotators may use binary, categorical, or continuous labels. Naturally, it is of interest to unify these labels from disparate scales to both achieve maximal coverage over words and to create a single, more robust sentiment lexicon while retaining scale coherence. We introduce a generative model of sentiment lexica to combine disparate scales into a common latent representation. We realize this model with a novel multi-view variational autoencoder (VAE), called SentiVAE. We evaluate our approach via a downstream text classification task involving nine English-Language sen- timent analysis datasets; our representation outperforms six individual sentiment lexica, as well as a straightforward combination thereof. 1 Introduction Sentiment lexica provide an easy way to automat- ically label texts with polarity values, and are also frequently transformed into features for supervised models, including neural networks (Palogiannidi et al., 2016; Ma et al., 2018). Indeed, given their utility, a veritable cottage industry has emerged focusing on the design of sentiment lexica. In prac- tice, using any single lexicon, unless specifically and carefully designed for the particular domain of interest, has several downsides. For example, any lexicon will typically have low coverage compared to the language’s entire vocabulary, and may have misspecified labels for the domain. In many cases, it may therefore be desirable to combine multiple sentiment lexica into a single representation. Indeed, some research on unifying Figure 1: A depiction of the “encoder” portion of Sen- tiVAE. The word peppy has polarity values of 0.65 and pos in the SenticNet and Hu-Liu lexica, respectively. These values are “encoded” into two three-dimensional vectors, which are then summed and added to (1, 1, 1) (not shown) to form the parameters of a Dirichlet over the latent representation of the word’s polarity value. such lexica has emerged (Emerson and Declerck, 2014; Altrabsheh et al., 2017), borrowing ideas from crowdsourcing (Raykar et al., 2010; Hovy et al., 2013). However, this is a non-trivial task, because lexica can use binary, categorical, or continuous scales to quantify polarity—in addition to different interpretations for each—and thus cannot easily be combined. In Fig. 1, we show an example of the same word labeled using different lexica to illustrate the nature of the challenge. To combine sentiment lexica with disparate scales, we introduce SentiVAE, a novel multi- view variant of the variational autoencoder (VAE) (Kingma and Welling, 2014). SentiVAE, visualized as a graphical model in Fig. …

Content truncated to 500 words