Neural Networks for Integrating Compositional and Non-compositional Sentiment in Sentiment Composition

Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics (*SEM 2015), pages 1–9, Denver, Colorado, June 4–5, 2015. Neural Networks for Integrating Compositional and Non-compositional Sentiment in Sentiment Composition Xiaodan Zhu & Hongyu Guo National Research Council Canada 1200 Montreal Road, M50 Ottawa, ON K1A 0R6, Canada {xiaodan.zhu,hongyu.guo}@nrc-cnrc.gc.ca Parinaz Sobhani EECS, University of Ottawa 800 King Edward Avenue Ottawa, ON K1N 6N5, Canada psobh090@uottawa.ca Abstract This paper proposes neural networks for inte- grating compositional and non-compositional sentiment in the process of sentiment compo- sition, a type of semantic composition that op- timizes a sentiment objective. We enable in- dividual composition operations in a recursive process to possess the capability of choosing and merging information from these two types of sources. We propose our models in neural network frameworks with structures, in which the merging parameters can be learned in a principled way to optimize a well-defined ob- jective. We conduct experiments on the Stan- ford Sentiment Treebank and show that the proposed models achieve better results over the model that lacks this ability. 1 Introduction Automatically determining the sentiment of a phrase, a sentence, or even a longer piece of text is still a challenging problem. Data sparseness en- countered in such tasks often requires to factorize the problem to consider smaller pieces of compo- nent words or phrases, for which much research has been performed on bag-of-words or bag-of-phrases models (Pang and Lee, 2008; Liu and Zhang, 2012). More recent work has started to model sentiment composition (Moilanen and Pulman, 2007; Choi and Cardie, 2008; Socher et al., 2012; Socher et al., 2013), a type of semantic composition that opti- mizes a sentiment objective. In general, the com- position process is critical in the formation of the sentiment of a span of text, which has not been well modeled yet and there is still scope for future work. Compositionality, or non-compositionality, of the senses of text spans is important for language under- standing. Sentiment, as one of the major semantic differential categories (Osgood et al., 1957), faces the problem as well. For example, the phrase must see or must try in a movie or restaurant review often indicates a positive sentiment, which, however, may be hard to learn from the component words. More extreme examples, e.g., slangs like bad ass, are not rare in social media text. This particular example can actually convey a very positive sentiment even though its component words are very negative. In brief, a sentiment composition framework that can consider both compositional and non-compositional sentiment is theoretically interesting. From a more pragmatical viewpoint, if one is able to reliably learn the sentiment of a text span (e.g., an ngram) holistically, it would be desirable that a composition model has the ability to de- cide the sources of knowledge it trusts more: the composition from the component words, the non- compositional source, or a soft combination of them. In such a situation, whether the text span is actually composable may be blur or may …

Content truncated to 500 words