Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–1642, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng and Christopher Potts Stanford University, Stanford, CA 94305, USA richard@socher.org,{aperelyg,jcchuang,ang}@cs.stanford.edu {jeaneis,manning,cgpotts}@stanford.edu Abstract Semantic word spaces have been very use- ful but cannot express the meaning of longer phrases in a principled way. Further progress towards understanding compositionality in tasks such as sentiment detection requires richer supervised training and evaluation re- sources and more powerful models of com- position. To remedy this, we introduce a Sentiment Treebank. It includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment composition- ality. To address them, we introduce the Recursive Neural Tensor Network. When trained on the new treebank, this model out- performs all previous methods on several met- rics. It pushes the state of the art in single sentence positive/negative classification from 80% up to 85.4%. The accuracy of predicting fine-grained sentiment labels for all phrases reaches 80.7%, an improvement of 9.7% over bag of features baselines. Lastly, it is the only model that can accurately capture the effects of negation and its scope at various tree levels for both positive and negative phrases. 1 Introduction Semantic vector spaces for single words have been widely used as features (Turney and Pantel, 2010). Because they cannot capture the meaning of longer phrases properly, compositionality in semantic vec- tor spaces has recently received a lot of attention (Mitchell and Lapata, 2010; Socher et al., 2010; Zanzotto et al., 2010; Yessenalina and Cardie, 2011; Socher et al., 2012; Grefenstette et al., 2013). How- ever, progress is held back by the current lack of large and labeled compositionality resources and – 0 0 This 0 film – – – 0 does 0 n’t 0 + care + 0 about + + + + + cleverness 0 , 0 wit 0 or + 0 0 any 0 0 other + kind + 0 of + + intelligent + + humor 0 . Figure 1: Example of the Recursive Neural Tensor Net- work accurately predicting 5 sentiment classes, very neg- ative to very positive (– –, –, 0, +, + +), at every node of a parse tree and capturing the negation and its scope in this sentence. models to accurately capture the underlying phe- nomena presented in such data. To address this need, we introduce the Stanford Sentiment Treebank and a powerful Recursive Neural Tensor Network that can accurately predict the compositional semantic effects present in this new corpus. The Stanford Sentiment Treebank is the first cor- pus with fully labeled parse trees that allows for a complete analysis of the compositional effects of sentiment in language. The corpus is based on the dataset introduced by Pang and Lee (2005) and consists of 11,855 single sentences extracted …

Content truncated to 500 words