CPH: Sentiment analysis of Figurative Language on Twitter #easypeasy #not

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 699–703, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics CPH: Sentiment analysis of Figurative Language on Twitter #easypeasy #not Sarah McGillion H´ector Mart´ınez Alonso Barbara Plank University of Copenhagen, Njalsgade 140, 2300 Copenhagen S, Denmark zhg159@alumni.ku.dk,alonso@hum.ku.dk,bplank@cst.dk Abstract This paper describes the details of our sys- tem submitted to the SemEval 2015 shared task on sentiment analysis of figurative lan- guage on Twitter. We tackle the problem as regression task and combine several base sys- tems using stacked generalization (Wolpert, 1992). An initial analysis revealed that the data is heavily biased, and a general sentiment analysis system (GSA) performs poorly on it. However, GSA proved helpful on the test data, which contains an estimated 25% non- figurative tweets. Our best system, a stacking system with backoff to GSA, ranked 4th on the final test data (Cosine 0.661, MSE 3.404).1 1 Introduction Sentiment analysis (SA) is the task of determining the sentiment of a given piece of text. The ampli- tude of user-generated content produced every day raises the importance of accurate automatic senti- ment analysis, for applications ranging from, e.g., reputation analysis (Amig´o et al., 2013) to election results prediction (Tjong Kim Sang and Bos, 2012). However, figurative language is pervasive in user- generated content, and figures of speech like irony, sarcasm and metaphors impose relevant challenges for a sentiment analysis system usually trained on literal meanings. For instance, consider the fol- lowing example:2 @CIA We hear you’re looking for sentiment analysis to detect sarcasm in Tweets. That’ll be easy! #SLA2014 #irony. Irony or sarcasm 1After submission time we discovered a bug in ST2,which means that the results on the official website are of the GSA and not of the stacking system with backoff. 2From the training data, label: -1.24; GSA prediction: +5. does not result always in the exact opposite senti- ment and therefore it is not as simple as just invert- ing the scores from a general SA system. Only few studies have attempted SA on figurative language so far (Reyes and Rosso, 2012; Reyes et al., 2013). The prediction of a fine-grained sentiment score (between -5 and 5) for a tweet poses a series of chal- lenges. First of all, accurate language technology on tweets is hard due to sample bias, i.e., collections of tweets are inherently biased towards the particular time (or way, cf. §2) they were collected (Eisenstein, 2013; Hovy et al., 2014). Secondly, the notion of figurativeness (or its complementary notion of liter- ality) does not have a strong definition, let alone do irony, sarcasm, or satire. As pointed out by Reyes and Rosso (2012), “there is not a clear distinction about the boundaries among these terms”. Yet alone attaching a fine-grained score is far from straight- forward. In fact, the gold standard consists of the average score assigned by humans through crowd- sourcing reflecting an uncertainty in ground truth. 2 Data Analysis The goal of the initial data exploration was …

Content truncated to 500 words