A Cognition Based Attention Model for Sentiment Analysis

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 462–471 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics A Cognition Based Attention Model for Sentiment Analysis Yunfei Long1, Qin Lu1, Rong Xiang2, Minglei Li1 and Chu-Ren Huang3 1Department of Computing, The Hong Kong Polytechnic University csylong,csluqin,csmli@comp.polyu.edu.hk 2Advanced Micro Devices, Inc.(Shanghai) Rong.Xiang@amd.com 3Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University churen.huang@polyu.edu.hk Abstract Attention models are proposed in senti- ment analysis because some words are more important than others. However, most existing methods either use local context based text information or user preference information. In this work, we propose a novel attention model trained by cognition grounded eye-tracking data. A reading prediction model is first built us- ing eye-tracking data as dependent data and other features in the context as in- dependent data. The predicted reading time is then used to build a cognition based attention (CBA) layer for neural sentiment analysis. As a comprehensive model, We can capture attentions of words in sentences as well as sentences in doc- uments. Different attention mechanisms can also be incorporated to capture other aspects of attentions. Evaluations show the CBA based method outperforms the state-of-the-art local context based atten- tion methods significantly. This brings in- sight to how cognition grounded data can be brought into NLP tasks. 1 Introduction Sentiment analysis is critical for many applica- tions such as sentimental product recommenda- tion (Dong et al., 2013), public opinion detec- tion (Pang et al., 2008), and human-machine inter- action (Clavel and Callejas, 2016), etc.Sentiment analysis has been well-explored (Pang et al., 2002; Vanzo et al., 2014; Tang et al., 2015a; Chen et al., 2016; Maas et al., 2011).Recently, deep learning based methods have further elevated the perfor- mance of sentiment analysis without the need for labor intensive feature engineering. Attention models are incorporated into senti- ment analysis because not all words are created equal. Some words are more important than oth- ers in conveying the message in a sentence. Simi- larly, some sentences are more important than oth- ers in a document. Although the overall reading time as a cognitive process may reflect the syn- tax and discourse complexity, reading time of in- dividual words is also an indicator of their seman- tic importance in text (Roseman, 2001; Demberg and Keller, 2008). Previous attention models are built using information embedded in text including users, products and text in local context for senti- ment classification (Tang et al., 2015b; Yang et al., 2016; Chen et al., 2016; Gui et al., 2016). How- ever, attention models using local context based text through distributional similarity lack theoret- ical foundation to reflect the cognitive basis. But, the key in sentiment analysis lies in its cognitive basis. Thus, we envision that cognition grounded data obtained in text reading should be helpful in building an attention model. In this paper, we propose a novel cognition based attention(CBA) model for sentiment analy- sis learned from cognition grounded eye-tracking data. Eye-tracking is the …

Content truncated to 500 words