Compositional Matrix-Space Models for Sentiment Analysis

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 172–182, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Compositional Matrix-Space Models for Sentiment Analysis Ainur Yessenalina Dept. of Computer Science Cornell University Ithaca, NY, 14853 ainur@cs.cornell.edu Claire Cardie Dept. of Computer Science Cornell University Ithaca, NY, 14853 cardie@cs.cornell.edu Abstract We present a general learning-based approach for phrase-level sentiment analysis that adopts an ordinal sentiment scale and is explicitly compositional in nature. Thus, we can model the compositional effects required for accu- rate assignment of phrase-level sentiment. For example, combining an adverb (e.g., “very”) with a positive polar adjective (e.g., “good”) produces a phrase (“very good”) with in- creased polarity over the adjective alone. In- spired by recent work on distributional ap- proaches to compositionality, we model each word as a matrix and combine words us- ing iterated matrix multiplication, which al- lows for the modeling of both additive and multiplicative semantic effects. Although the multiplication-based matrix-space framework has been shown to be a theoretically ele- gant way to model composition (Rudolph and Giesbrecht, 2010), training such models has to be done carefully: the optimization is non- convex and requires a good initial starting point. This paper presents the first such al- gorithm for learning a matrix-space model for semantic composition. In the context of the phrase-level sentiment analysis task, our ex- perimental results show statistically signifi- cant improvements in performance over a bag- of-words model. 1 Introduction Sentiment analysis has been an active research area in recent years. Work in the area ranges from iden- tifying the sentiment of individual words to deter- mining the sentiment of phrases, sentences and doc- uments (see Pang and Lee (2008) for a survey). The bulk of previous research, however, models just pos- itive vs. negative sentiment, collapsing positive (or negative) words, phrases and documents of differ- ing intensities into just one positive (or negative) class. For word-level sentiment, therefore, these methods would not recognize a difference in senti- ment between words like “good” and “great”, which have the same direction of polarity (i.e., positive) but different intensities. At the phrase level, the methods will fail to register compositional effects in sentiment brought about by intensifiers like “very”, “absolutely”, “extremely”, etc. “Happy” and “very happy”, for example, will both be considered sim- ply “positive” in sentiment. In real-world settings, on the other hand, sentiment values extend across a polarity spectrum — from very negative, to neutral, to very positive. Recent research has shown, in par- ticular, that modeling intensity at the phrase level is important for real-world natural language process- ing tasks including question answering and textual entailment (de Marneffe et al., 2010). This paper describes a general approach for phrase-level sentiment analysis that takes these real- world requirements into account: we adopt a five- level ordinal sentiment scale and present a learning- based method that assigns ordinal sentiment scores to phrases. Importantly, our approach will also be explicitly compositional1 in nature so that it can accurately …

Content truncated to 500 words