Latent Variable Sentiment Grammar

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4642–4651 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4642 Latent Variable Sentiment Grammar∗ Liwen Zhang†, Kewei Tu†, Yue Zhang‡⋄ †School of Information Science and Technology,ShanghaiTech University, Shanghai, China ‡Institute of Advanced Technology, Westlake Institute for Advanced Study, China ⋄School of Engineering, Westlake University, Hangzhou, China {zhanglw1,tukw}@shanghaitech.edu.cn yuezhang@westlake.edu.cn Abstract Neural models have been investigated for sentiment classification over constituent trees. They learn phrase composition automatically by encoding tree structures but do not explic- itly model sentiment composition, which re- quires to encode sentiment class labels. To this end, we investigate two formalisms with deep sentiment representations that capture sentiment subtype expressions by latent vari- ables and Gaussian mixture vectors, respec- tively. Experiments on Stanford Sentiment Treebank (SST) show the effectiveness of sen- timent grammar over vanilla neural encoders. Using ELMo embeddings, our method gives the best results on this benchmark. 1 Introduction Determining the sentiment polarity at or below the sentence level is an important task in natural lan- guage processing. Sequence structured models (Li et al., 2015; McCann et al., 2017) have been ex- ploited for modeling each phrase independently. Recently, tree structured models (Zhu et al., 2015; Tai et al., 2015; Teng and Zhang, 2017) were lever- aged for learning phrase compositions in sentence representation given the syntactic structure. Such models classify the sentiment over each constituent node according to its hidden vector through tree structure encoding. Though effective, existing neural methods do not consider explicit sentiment compositionality (Mon- tague, 1974). Take the sentence “The movie is not very good, but I still like it” in Figure 1 as example (Dong et al., 2015), over the constituent tree, sen- timent signals can be propagated from leaf nodes to the root, going through negation, intensification and contrast according to the context. Modeling such signal channels can intuitively lead to more ∗ Work was done when the first author was visiting West- lake University. The third author is the corresponding author. Draw tree guiwuqiansha March 2019 1 Introduction 1 -1 -1 0 The movie is -1 -1 not 1 0 very 1 good 0 , 0 but 1 I still like it 1 Figure 1: Example of sentiment composition interpretable and reliable results. To model sen- timent composition, direct encoding of sentiment signals (e.g., +1/-1 or more fine-grained forms) is necessary. To this end, we consider a neural network gram- mar with latent variables. In particular, we em- ploy a grammar as the backbone of our approach in which nonterminals represent sentiment signals and grammar rules specify sentiment compositions. In the simplest version of our approach, nonterminals are sentiment labels from SST directly, resulting in a weighted grammar. To model more fine-grained emotions (Ortony and Turner, 1990), we consider a latent variable grammar (LVG, Matsuzaki et al. (2005), Petrov et al. (2006)), which splits each non- terminal into subtypes to represent subtle sentiment signals and uses a discrete latent variable to denote …

Content truncated to 500 words