Emotion Intensities in Tweets

Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), pages 65–77, Vancouver, Canada, August 3-4, 2017. c⃝2017 Association for Computational Linguistics Emotion Intensities in Tweets Saif M. Mohammad Information and Communications Technologies National Research Council Canada Ottawa, Canada saif.mohammad@nrc-cnrc.gc.ca Felipe Bravo-Marquez Department of Computer Science The University of Waikato Hamilton, New Zealand fbravoma@waikato.ac.nz Abstract This paper examines the task of detecting intensity of emotion from text. We cre- ate the first datasets of tweets annotated for anger, fear, joy, and sadness intensities. We use a technique called best–worst scal- ing (BWS) that improves annotation con- sistency and obtains reliable fine-grained scores. We show that emotion-word hash- tags often impact emotion intensity, usu- ally conveying a more intense emotion. Fi- nally, we create a benchmark regression system and conduct experiments to deter- mine: which features are useful for detect- ing emotion intensity; and, the extent to which two emotions are similar in terms of how they manifest in language. 1 Introduction We use language to communicate not only the emotion we are feeling but also the intensity of the emotion. For example, our utterances can con- vey that we are very angry, slightly sad, absolutely elated, etc. Here, intensity refers to the degree or amount of an emotion such as anger or sad- ness.1 Natural language applications can benefit from knowing both the class of emotion and its intensity. For example, a commercial customer satisfaction system would prefer to focus first on instances of significant frustration or anger, as op- posed to instances of minor inconvenience. How- ever, most work on automatic emotion detection has focused on categorical classification (presence of anger, joy, sadness, etc.). A notable obstacle in developing automatic affect intensity systems is the lack of suitable annotated data. Existing af- fect datasets are predominantly categorical. Anno- 1Intensity is different from arousal, which refers to the extent to which an emotion is calming or exciting. tating instances for degrees of affect is a substan- tially more difficult undertaking: respondents are presented with greater cognitive load and it is par- ticularly hard to ensure consistency (both across responses by different annotators and within the responses produced by an individual annotator). Best–Worst Scaling (BWS) is an annotation scheme that addresses these limitations (Louviere, 1991; Louviere et al., 2015; Kiritchenko and Mo- hammad, 2016, 2017). Annotators are given n items (an n-tuple, where n > 1 and commonly n = 4). They are asked which item is the best (highest in terms of the property of inter- est) and which is the worst (lowest in terms of the property of interest). When working on 4- tuples, best–worst annotations are particularly ef- ficient because each best and worst annotation will reveal the order of five of the six item pairs. For example, for a 4-tuple with items A, B, C, and D, if A is the best, and D is the worst, then A > B, A > C, A > D, B > D, and C > D. …

Content truncated to 500 words