Cascading Multiway Attentions for Document-level Sentiment Classification

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 634–643, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Cascading Multiway Attention for Document-level Sentiment Classification Dehong Ma, Sujian Li, Xiaodong Zhang, Houfeng Wang, Xu Sun MOE Key Lab of Computational Linguistics, Peking University, Beijing 100871, China {madehong, lisujian, zxdcs, wanghf, xusun}@pku.edu.cn Abstract Document-level sentiment classification aims to assign the user reviews a sentiment polarity. Previous methods either just uti- lized the document content without con- sideration of user and product information, or did not comprehensively consider what roles the three kinds of information play in text modeling. In this paper, to reasonably use all the information, we present the idea that user, product and their combination can all influence the generation of atten- tion to words and sentences, when judg- ing the sentiment of a document. With this idea, we propose a cascading multiway attention (CMA) model, where multiple ways of using user and product informa- tion are cascaded to influence the genera- tion of attention on the word and sentence layers. Then, sentences and documents are well modeled by multiple representa- tion vectors, which provide rich informa- tion for sentiment classification. Experi- ments on IMDB and Yelp datasets demon- strate the effectiveness of our model. 1 Introduction Document-level sentiment classification aims to predict an overall sentiment polarity (e.g., 1-5 stars or 1-10 stars) for a user review document. This task recently draws increasing research concerns and is helpful to many downstream applications, such as user and product recommendation. Early work focuses on traditional machine learning associated with handcraft text features for sentiment classification (Pang et al., 2002; Ding et al., 2008; Taboada et al., 2011). With the development of deep learning techniques, some researchers design neural networks to automati- cally learn features from document content and achieve comparable performance (Glorot et al., 2011; Kalchbrenner et al., 2014; Yang et al., 2016), though they ignore the use of user and product information. Almost at the same time, deep learning techniques exhibit another advan- tage that product and user information can be flex- ibly modeled with document content for senti- ment classification (Tang et al., 2015a; Chen et al., 2016). Tang et al. (2015a) design user and prod- uct preference matrices to tune word representa- tions, based on which convolutional neural net- works (CNNs) are used to model the whole doc- ument. To avoid the high-cost preference ma- trix, Chen et al. (2016) develop the two-layer (i.e., word and sentence layers) model, where the com- bination of user and product information is used to generate attention to words and sentences respec- tively on each layer. Though previous studies achieve improvements in synthesizing text, user and product for senti- ment classification, they are somewhat limited to either of the following two aspects. First, when considering user and product information, previ- ous work mostly models their combination which may cause adverse effects on sentiment classifi- cation. As we can see, user and product infor- mation also have …

Content truncated to 500 words