CrystalNest at SemEval-2017 Task 4: Using Sarcasm Detection for Enhancing Sentiment Classification and Quantification

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 626–633, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics CrystalNest at SemEval-2017 Task 4: Using Sarcasm Detection for Enhancing Sentiment Classification and Quantification Raj Kumar Gupta and Yinping Yang∗ Institute of High Performance Computing (IHPC) Agency for Science, Technology and Research (A*STAR), Singapore {gupta-rk, yangyp}@ihpc.a-star.edu.sg Abstract This paper describes a system developed for a shared sentiment analysis task and its subtasks organized by SemEval-2017. A key feature of our system is the em- bedded ability to detect sarcasm in order to enhance the performance of sentiment classification. We first constructed an affect-cognition-sociolinguistics sarcasm features model and trained a SVM-based classifier for detecting sarcastic expres- sions from general tweets. For sentiment prediction, we developed CrystalNest—a two-level cascade classification system us- ing features combining sarcasm score de- rived from our sarcasm classifier, senti- ment scores from Alchemy, NRC lexi- con, n-grams, word embedding vectors, and part-of-speech features. We found that the sarcasm detection derived features consistently benefited key sentiment anal- ysis evaluation metrics, in different de- grees, across four subtasks A-D. 1 Introduction Sentiment analysis, also known as opinion mining, is the study of the feelings and opinions from user- generated content. Sarcasm detection, though very related, is a different topic of interest. As a clas- sification task, the primary objective of sentiment analysis is to determine if a message is positive, negative, or neutral. In contrast, the objective of sarcasm detection is to determine if a message is sarcastic or not sarcastic. To illustrate, let us look at two short text ex- amples. Example 1 expresses a positive sentiment ∗Both authors contributed to this research equally. For correspondence, please contact yangyp@ihpc.a-star.edu.sg. which has a slight mixed feeling, but it is not sar- castic. A very similar-looking Example 2 is sar- castic, and its underlying sentiment is negative. Ex 1. Love my new phone! Only that the battery runs out very fast. Ex 2. Love my new phone that runs out battery so fast! In computational linguistics and NLP, detecting sarcasm is receiving increasing research interest (e.g., Gonz´alez-Ib´a˜nez et al., 2011; Reyes et al., 2012; Liebrecht et al., 2013; Riloff et al., 2013; Rajadesingan et al., 2015; Bamman and Smith, 2015). While these studies recognized the linkage between sarcasm and sentiment and have proposed various techniques for detecting sarcasm, none directly studied the impact of sarcasm detection on sentiment analysis. Maynard and Greenwood (2014) is among the first to explore how to use sarcasm-related information to improve sentiment analysis. They proposed a rule-based method in- volving five rules such as using “#sarcasm” to flip a sentiment from positive to negative. How- ever, their evaluation was performed on a rela- tively small test dataset of 400 tweets. We believe that sentiment analysis systems will benefit from a systematically embedded ability to detect sarcasm. In the following, we describe our approach and present supportive findings eval- uated on a large set of test data provided by SemEval-2017 Task 4 …

Content truncated to 500 words