Neural Sentiment Classification with User and Product Attention

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1650–1659, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Neural Sentiment Classification with User and Product Attention Huimin Chen1, Maosong Sun1,2∗, Cunchao Tu1, Yankai Lin1, Zhiyuan Liu1 1Department of Computer Science and Technology, State Key Lab on Intelligent Technology and Systems, National Lab for Information Science and Technology, Tsinghua University, Beijing, China 2Beijing Advanced Innovation Center for Imaging Technology, Capital Normal University, Beijing, China Abstract Document-level sentiment classification aims to predict user’s overall sentiment in a doc- ument about a product. However, most of existing methods only focus on local text in- formation and ignore the global user pref- erence and product characteristics. Even though some works take such information into account, they usually suffer from high model complexity and only consider word- level preference rather than semantic levels. To address this issue, we propose a hierarchi- cal neural network to incorporate global user and product information into sentiment clas- sification. Our model first builds a hierar- chical LSTM model to generate sentence and document representations. Afterwards, user and product information is considered via at- tentions over different semantic levels due to its ability of capturing crucial semantic com- ponents. The experimental results show that our model achieves significant and consistent improvements compared to all state-of-the- art methods. The source code of this paper can be obtained from https://github. com/thunlp/NSC. 1 Introduction Sentiment analysis aims to analyze people’s sen- timents or opinions according to their generated texts and plays a critical role in the area of data mining and natural language processing. Recently, sentiment analysis draws increasing attention of re- searchers with the rapid growth of online review ∗Corresponding author: M. Sun (sms@tsinghua.edu.cn) sites such as Amazon, Yelp and IMDB, due to its importance to personalized recommendation. In this work, we focus on the task of document- level sentiment classification, which is a fundamen- tal problem of sentiment analysis. Document-level sentiment classification assumes that each docu- ment expresses a sentiment on a single product and targets to determine the overall sentiment about the product. Most existing methods take sentiment classifica- tion as a special case of text classification problem. Such methods treat annotated sentiment polarities or ratings as categories and apply machine learning algorithms to train classifiers with text features, e.g., bag-of-words vectors (Pang et al., 2002). Since the performance of text classifiers heavily depends on the extracted features, such studies usually attend to design effective features from text or additional sen- timent lexicons (Ding et al., 2008; Taboada et al., 2011). Motivated by the successful utilization of deep neural networks in computer vision (Ciresan et al., 2012), speech recognition (Dahl et al., 2012) and natural language processing (Bengio et al., 2006), some neural network based sentiment analysis mod- els are proposed to learn low-dimensional text fea- tures without any feature engineering (Glorot et al., 2011; Socher et al., 2011; Socher et al., 2012; Socher et al., 2013; Kim, 2014). Most proposed neural network models …

Content truncated to 500 words