Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 151–161, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions Richard Socher Jeffrey Pennington∗ Eric H. Huang Andrew Y. Ng Christopher D. Manning Computer Science Department, Stanford University, Stanford, CA 94305, USA ∗SLAC National Accelerator Laboratory, Stanford University, Stanford, CA 94309, USA richard@socher.org {jpennin,ehhuang,ang,manning}@stanford.edu ang@cs.stanford.edu Abstract We introduce a novel machine learning frame- work based on recursive autoencoders for sentence-level prediction of sentiment label distributions. Our method learns vector space representations for multi-word phrases. In sentiment prediction tasks these represen- tations outperform other state-of-the-art ap- proaches on commonly used datasets, such as movie reviews, without using any pre-defined sentiment lexica or polarity shifting rules. We also evaluate the model’s ability to predict sentiment distributions on a new dataset based on confessions from the experience project. The dataset consists of personal user stories annotated with multiple labels which, when aggregated, form a multinomial distribution that captures emotional reactions. Our al- gorithm can more accurately predict distri- butions over such labels compared to several competitive baselines. 1 Introduction The ability to identify sentiments about personal ex- periences, products, movies etc. is crucial to un- derstand user generated content in social networks, blogs or product reviews. Detecting sentiment in these data is a challenging task which has recently spawned a lot of interest (Pang and Lee, 2008). Current baseline methods often use bag-of-words representations which cannot properly capture more complex linguistic phenomena in sentiment analy- sis (Pang et al., 2002). For instance, while the two phrases “white blood cells destroying an infection” and “an infection destroying white blood cells” have the same bag-of-words representation, the former is a positive reaction while the later is very negative. More advanced methods such as (Nakagawa et al., Indices Words Semantic Representations Recursive Autoencoder i walked into a parked car Sorry, Hugs You Rock Teehee I Understand Wow, Just Wow Predicted Sentiment Distribution Figure 1: Illustration of our recursive autoencoder archi- tecture which learns semantic vector representations of phrases. Word indices (orange) are first mapped into a semantic vector space (blue). Then they are recursively merged by the same autoencoder network into a fixed length sentence representation. The vectors at each node are used as features to predict a distribution over senti- ment labels. 2010) that can capture such phenomena use many manually constructed resources (sentiment lexica, parsers, polarity-shifting rules). This limits the ap- plicability of these methods to a broader range of tasks and languages. Lastly, almost all previous work is based on single, positive/negative categories or scales such as star ratings. Examples are movie reviews (Pang and Lee, 2005), opinions (Wiebe et al., 2005), customer reviews (Ding et al., 2008) or multiple aspects of restaurants (Snyder and Barzilay, 2007). Such a one-dimensional scale does not accu- rately reflect the complexity of human emotions and sentiments. In this work, we seek to address three issues. (i) Instead of using a bag-of-words representation, our model …

Content truncated to 500 words