SemEval-2016 Task 7: Determining Sentiment Intensity of English and Arabic Phrases

Proceedings of SemEval-2016, pages 42–51, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics SemEval-2016 Task 7: Determining Sentiment Intensity of English and Arabic Phrases Svetlana Kiritchenko and Saif M. Mohammad National Research Council Canada svetlana.kiritchenko@nrc-cnrc.gc.ca saif.mohammad@nrc-cnrc.gc.ca Mohammad Salameh University of Alberta msalameh@ualberta.ca Abstract We present a shared task on automatically de- termining sentiment intensity of a word or a phrase. The words and phrases are taken from three domains: general English, English Twit- ter, and Arabic Twitter. The phrases include those composed of negators, modals, and de- gree adverbs as well as phrases formed by words with opposing polarities. For each of the three domains, we assembled the datasets that include multi-word phrases and their con- stituent words, both manually annotated for real-valued sentiment intensity scores. The three datasets were presented as the test sets for three separate tasks (each focusing on a specific domain). Five teams submitted nine system outputs for the three tasks. All datasets created for this shared task are freely available to the research community. 1 Introduction Words have prior associations with sentiment. For example, honest and competent are associated with positive sentiment, whereas dishonest and dull are associated with negative sentiment. Further, the de- gree of positivity (or negativity), also referred to as intensity, can vary. For example, most people will agree that succeed is more positive (or less negative) than improve, and fail is more negative (or less pos- itive) than setback. We present a shared task where automatic systems are asked to predict a prior sen- timent intensity score for a word or a phrase. The words and phrases are taken from three domains: general English, English Twitter, and Arabic Twitter. For each domain, a separate task with its own devel- opment and test sets was set up. The phrases include those composed of negators (e.g., nothing wrong), modals (e.g., might be fun), and degree adverbs (e.g., fairly important) as well as phrases formed by words with opposing polarities (e.g., lazy sundays). Lists of words and their associated sentiment are commonly referred to as sentiment lexicons. They are used in sentiment analysis. For example, a number of unsupervised classifiers rely primarily on sentiment lexicons to determine whether a piece of text is positive or negative. Supervised classifiers also often use features drawn from sentiment lexi- cons (Mohammad et al., 2013; Pontiki et al., 2014). Sentiment lexicons are also beneficial in stance de- tection (Mohammad et al., 2016a; Mohammad et al., 2016b), literary analysis (Hartner, 2013; Kleres, 2011; Mohammad, 2012), and for detecting person- ality traits (Minamikawa and Yokoyama, 2011; Mo- hammad and Kiritchenko, 2015). Existing manually created sentiment lexicons tend to provide only lists of positive and negative words (Hu and Liu, 2004; Wilson et al., 2005; Mo- hammad and Turney, 2013). The coarse-grained distinctions may be less useful in downstream ap- plications than having access to fine-grained (real- valued) sentiment association scores. Most of the existing sentiment resources are available only for English. Non-English resources are scarce and often …

Content truncated to 500 words