Sentence and Clause Level Emotion Annotation, Detection, and Classification in a Multi-Genre Corpus

Sentence and Clause Level Emotion Annotation, Detection, and Classification in a Multi-Genre Corpus Shabnam Tafreshi, Mona Diab George Washington University Department of Computer Science, Washington D.C. {shabnamt, mtdiab}@gwu.edu Abstract Predicting emotion categories (e.g. anger, joy, sadness) expressed by a sentence is challenging due to inherent multi-label smaller pieces such as phrases and clauses. To date, emotion has been studied in single genre, while models of human behaviors or situational awareness in the event of disasters require emotion modeling in multi-genres. In this paper, we expand and unify existing annotated data in different genres (emotional blog post, news title, and movie reviews) using an inventory of 8 emotions from Plutchik’s Wheel of Emotions tags. We develop systems for automatically detecting and classifying emotions in text, in different textual genres and granularity levels, namely, sentence and clause levels in a supervised setting. We explore the effectiveness of clause annotation in sentence-level emotion detection and classification (EDC). To our knowledge, our EDC system is the first to target the clause level; further we provide emotion annotation for movie reviews dataset for the first time. Keywords: emotion annotation, emotion analysis, multi-genre corpus 1. Introduction Prediction of sentence-level emotion classification encom- pass a variety of applications such as modeling of hu- man behaviors (Dodds and Danforth, 2010) and situational awareness in the event of disasters (Vo and Collier, 2013). As a precursor to our system development, we realize the diversity and non uniformity of existing resources with emotion tags, hence, we re-annotate existing resources in a unified framework, thereby covering multiple genres of text. The genres are as follows: emotional blog post (BLG), news headlines dataset (HLN), and movie review dataset (MOV). We present an approach and system that performs emotion detection and classification (EDC) on multiple lev- els of granularity, namely, sentence and clause levels. We expand the annotation scheme to cover both sentence and clause level annotations, as well as expand the emotion tag inventory from the typical Ekman 6 (Ekman, 1992) emo- tion labels (EK6) to 8 emotion labels based on Plutchik’s Wheel of Emotions (Plutchik, 1962) (PL8). In this study, we focus on the impact of clause-level anno- tation on the EDC task, which can be used effectively in a single-genre or multi-genre textual setting without signifi- cant performance loss. Similar to previous studies, we cast the EDC problem in a supervised setting. Evaluation of EDC in 10% held out data outperformed the baseline and gives the average accuracy of 81.1% and 71.3% for sen- tence and clause level respectively. EDC achieved better results compared to previous annotation of HLN and BLG datasets with EK6 emotion labels (average accuracy 54.7% and 73.8%). Accordingly, our contributions are as follows: • A new set of annotation guidelines for emotion detec- tion based on Plutchik’s Wheel of Emotions. • A uniformly annotated multi-genre data set (including old and new data) on two levels of granularity: sen- tence and clause levels. • Two EDC systems on the sentence and clause levels for multiple genres leveraging …

Content truncated to 500 words