SemEval-2017 Task 4: Sentiment Analysis in Twitter

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 502–518, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics SemEval-2017 Task 4: Sentiment Analysis in Twitter Sara Rosenthal♣, Noura Farra♦, Preslav Nakov♥ ♥Qatar Computing Research Institute, Hamad bin Khalifa University, Qatar ♦Department of Computer Science, Columbia University ♣IBM Research, USA Abstract This paper describes the fifth year of the Sentiment Analysis in Twitter task. SemEval-2017 Task 4 continues with a rerun of the subtasks of SemEval-2016 Task 4, which include identifying the over- all sentiment of the tweet, sentiment to- wards a topic with classification on a two- point and on a five-point ordinal scale, and quantification of the distribution of sen- timent towards a topic across a number of tweets: again on a two-point and on a five-point ordinal scale. Compared to 2016, we made two changes: (i) we in- troduced a new language, Arabic, for all subtasks, and (ii) we made available in- formation from the profiles of the Twitter users who posted the target tweets. The task continues to be very popular, with a total of 48 teams participating this year. 1 Introduction The identification of sentiment in text is an im- portant field of study, with social media plat- forms such as Twitter garnering the interest of re- searchers in language processing as well as in po- litical and social sciences. The task usually in- volves detecting whether a piece of text expresses a POSITIVE, a NEGATIVE, or a NEUTRAL senti- ment; the sentiment can be general or about a spe- cific topic, e.g., a person, a product, or an event. The Sentiment Analysis in Twitter task has been run yearly at SemEval since 2013 (Nakov et al., 2013; Rosenthal et al., 2014; Nakov et al., 2016b), with the 2015 task introducing sentiment towards a topic (Rosenthal et al., 2015) and the 2016 task introducing tweet quantification and five-point or- dinal classification (Nakov et al., 2016a). SemEval is the International Workshop on Seman- tic Evaluation, formerly SensEval. It is an on- going series of evaluations of computational se- mantic analysis systems, organized under the um- brella of SIGLEX, the Special Interest Group on the Lexicon of the Association for Computational Linguistics. Other related tasks at SemEval have explored sentiment analysis of product review and their aspects (Pontiki et al., 2014, 2015, 2016), sentiment analysis of figurative language on Twit- ter (Ghosh et al., 2015), implicit event polarity (Russo et al., 2015), detecting stance in tweets (Mohammad et al., 2016a), out-of-context senti- ment intensity of words and phrases (Kiritchenko et al., 2016), and emotion detection (Strapparava and Mihalcea, 2007). Some of these tasks fea- tured languages other than English, such as Arabic (Pontiki et al., 2016; Mohammad et al., 2016a); however, they did not target tweets, nor did they focus on sentiment towards a topic. This year, we performed a re-run of the subtasks in SemEval-2016 Task 4, which, in addition to the overall sentiment of a tweet, featured classifica- tion, ordinal …

Content truncated to 500 words