EmoTweet-28: A Fine-Grained Emotion Corpus for Sentiment Analysis

EmoTweet-28: A Fine-Grained Emotion Corpus for Sentiment Analysis Jasy Liew Suet Yan, Howard R. Turtle, Elizabeth D. Liddy School of Information Studies, Syracuse University Syracuse, New York, USA E-mail: jliewsue@syr.edu, turtle@syr.edu, liddy@syr.edu Abstract This paper describes EmoTweet-28, a carefully curated corpus of 15,553 tweets annotated with 28 emotion categories for the purpose of training and evaluating machine learning models for emotion classification. EmoTweet-28 is, to date, the largest tweet corpus annotated with fine-grained emotion categories. The corpus contains annotations for four facets of emotion: valence, arousal, emotion category and emotion cues. We first used small-scale content analysis to inductively identify a set of emotion categories that characterize the emotions expressed in microblog text. We then expanded the size of the corpus using crowdsourcing. The corpus encompasses a variety of examples including explicit and implicit expressions of emotions as well as tweets containing multiple emotions. EmoTweet-28 represents an important resource to advance the development and evaluation of more emotion-sensitive systems. Keywords: emotion corpus, sentiment analysis, microblog text 1. Introduction Twitter, a popular microblogging site, provides a window into the emotional worlds of significant user populations. Twitter data can be leveraged to study behavior on social media in a non-intrusive manner. Given the large traffic volume (500 million tweets per day), automatic techniques to detect emotions are needed to augment our ability to analyze and understand emotion content. Automatic emotion detection in text at a fine-grained level is potentially useful for applications such as personality detection, public and behavioral health monitoring as well as consumer and market analysis. An important starting point for building natural language processing (NLP) systems to detect emotions in text is the construction of a ground truth corpus annotated to characterize emotion content. Existing emotion corpora are annotated with relatively coarse-grained categories (e.g., 6 to 8 basic emotion categories) (Mohammad, Zhu, & Martin, 2014; Roberts et al., 2012). Many current corpora are automatically annotated based on emotion hashtags rather than human judgment (Mohammad, 2012; Pak & Paroubek, 2010; Wang et al., 2012). With the basic emotions (happiness, sadness, fear, anger, disgust and surprise) accepted as the state-of-the-art, existing emotion corpora and other language resources that serve as the basis for building and evaluating mechanisms to detect emotion in tweets are only annotated with those basic categories. As a result, automatic emotion detectors developed using these resources are only able to give us a limited picture of human emotion expressions. Other emotions apart from the basic set, as well as variations within each basic emotion are “virgin territories” that have not yet been explored by researchers in this area. Efforts to increase the utility of automatic emotion detectors have to start with extending language resources to cover other emotion categories. We describe our efforts to construct a gold standard corpus of 15,553 tweets annotated with 28 emotion categories for the purpose of training and evaluating machine learning models for emotion classification. EmoTweet-28 is the largest carefully curated tweet corpus annotated with fine-grained emotion categories. The contributions of the paper …

Content truncated to 500 words