Learning Cognitive Features from Gaze Data for Sentiment and Sarcasm Classification using Convolutional Neural Network

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 377–387 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1035 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 377–387 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1035 Learning Cognitive Features from Gaze Data for Sentiment and Sarcasm Classification using Convolutional Neural Network Abhijit Mishra†, Kuntal Dey†, Pushpak Bhattacharyya⋆ †IBM Research, India ⋆Indian Institute of Technology Bombay, India †{abhijimi, kuntadey}@in.ibm.com ⋆pb@cse.iitb.ac.in Abstract Cognitive NLP systems- i.e., NLP systems that make use of behavioral data - augment traditional text-based features with cogni- tive features extracted from eye-movement patterns, EEG signals, brain-imaging etc.. Such extraction of features is typically manual. We contend that manual extrac- tion of features may not be the best way to tackle text subtleties that characteristically prevail in complex classification tasks like sentiment analysis and sarcasm detection, and that even the extraction and choice of features should be delegated to the learn- ing system. We introduce a framework to automatically extract cognitive features from the eye-movement / gaze data of hu- man readers reading the text and use them as features along with textual features for the tasks of sentiment polarity and sar- casm detection. Our proposed framework is based on Convolutional Neural Network (CNN). The CNN learns features from both gaze and text and uses them to clas- sify the input text. We test our technique on published sentiment and sarcasm la- beled datasets, enriched with gaze infor- mation, to show that using a combination of automatically learned text and gaze fea- tures often yields better classification per- formance over (i) CNN based systems that rely on text input alone and (ii) existing systems that rely on handcrafted gaze and textual features. 1 Introduction Detection of sentiment and sarcasm in user- generated short reviews is of primary importance for social media analysis, recommendation and di- alog systems. Traditional sentiment analyzers and sarcasm detectors face challenges that arise at lex- ical, syntactic, semantic and pragmatic levels (Liu and Zhang, 2012; Mishra et al., 2016c). Feature- based systems (Akkaya et al., 2009; Sharma and Bhattacharyya, 2013; Poria et al., 2014) can aptly handle lexical and syntactic challenges (e.g. learn- ing that the word deadly conveys a strong positive sentiment in opinions such as Shane Warne is a deadly bowler, as opposed to The high altitude Hi- malayan roads have deadly turns). It is, however, extremely difficult to tackle subtleties at semantic and pragmatic levels. For example, the sentence I really love my job. I work 40 hours a week to be this poor. requires an NLP system to be able to understand that the opinion holder has not ex- pressed a positive sentiment towards her / his job. In the absence of explicit clues in the text, it is dif- ficult for automatic systems to arrive at a correct classification decision, as they often lack external knowledge about various aspects of …

Content truncated to 500 words