Understanding Emotions: A Dataset of Tweets to Study Interactions between Affect Categories

Understanding Emotions: A Dataset of Tweets to Study Interactions between Affect Categories Saif M. Mohammad and Svetlana Kiritchenko National Research Council Canada {saif.mohammad,svetlana.kiritchenko}@nrc-cnrc.gc.ca Abstract Human emotions are complex and nuanced. Yet, an overwhelming majority of the work in automatically detecting emotions from text has focused only on classifying text into positive, negative, and neutral classes, and a much smaller amount on classifying text into basic emotion categories such as joy, sadness, and fear. Our goal is to create a single textual dataset that is annotated for many emotion (or affect) dimensions (from both the basic emotion model and the VAD model). For each emotion dimension, we annotate the data for not just coarse classes (such as anger or no anger) but also for fine-grained real-valued scores indicating the intensity of emotion (anger, sadness, valence, etc.). We use Best–Worst Scaling (BWS) to address the limitations of traditional rating scale methods such as inter- and intra-annotator inconsistency by employing comparative annotations. We show that the fine-grained intensity scores thus obtained are reliable (repeat annotations lead to similar scores). We choose Twitter as the source of the textual data we annotate because tweets are self-contained, widely used, public posts, and tend to be rich in emotions. The new dataset is useful for training and testing supervised machine learning algorithms for multi-label emotion classification, emotion intensity regression, detecting valence, detecting ordinal class of intensity of emotion (slightly sad, very angry, etc.), and detecting ordinal class of valence (or sentiment). We make the data available for the recent SemEval-2018 Task 1: Affect in Tweets, which explores these five tasks. The dataset also sheds light on crucial research questions such as: which emotions often present together in tweets?; how do the intensities of the three negative emotions relate to each other?; and how do the intensities of the basic emotions relate to valence? Keywords: emotion intensity, valence, arousal, dominance, basic emotions, crowdsourcing, sentiment analysis 1. Introduction Emotions are central to how we perceive the world, how we make sense of it, and how we make day-to-day de- cisions. Emotions are also complex and nuanced. Even though humans are known to perceive hundreds of differ- ent emotions, there is still little agreement on how best to categorize and represent emotions. According to the basic emotion model (aka the categorical model) (Ekman, 1992; Plutchik, 1980; Parrot, 2001; Frijda, 1988), some emotions, such as joy, sadness, fear, etc., are more basic than others, and that these emotions are each to be treated as separate categories. Each of these emotions can be felt or expressed in varying intensities. Here, intensity refers to the degree or amount of an emotion such as anger or sadness.1 As per the valence–arousal–dominance (VAD) model (Russell, 2003), emotions are points in a three-dimensional space of valence (positiveness–negativeness), arousal (active–passive), and dominance (dominant–submissive). Both the categorical model and the dimensional model of emotions have a large body of work supporting them, and offer different perspectives that help our understanding of emotions. However, there is very little …

Content truncated to 500 words