Crowdsourcing and Validating Event-focused Emotion Corpora for German and English

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4005–4011 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4005 Crowdsourcing and Validating Event-focused Emotion Corpora for German and English Enrica Troiano, Sebastian Pad´o and Roman Klinger Institut f¨ur Maschinelle Sprachverarbeitung University of Stuttgart, Germany {firstname.lastname}@ims.uni-stuttgart.de Abstract Sentiment analysis has a range of corpora available across multiple languages. For emo- tion analysis, the situation is more limited, which hinders potential research on cross- lingual modeling and the development of pre- dictive models for other languages. In this pa- per, we fill this gap for German by construct- ing deISEAR, a corpus designed in analogy to the well-established English ISEAR emo- tion dataset. Motivated by Scherer’s appraisal theory, we implement a crowdsourcing exper- iment which consists of two steps. In step 1, participants create descriptions of emotional events for a given emotion. In step 2, five an- notators assess the emotion expressed by the texts. We show that transferring an emotion classification model from the original English ISEAR to the German crowdsourced deISEAR via machine translation does not, on average, cause a performance drop. 1 Introduction Feeling emotions is a central part of the “human condition” (Russell, 1945). While existing stud- ies on automatic recognition of emotions in text have achieved promising results (Pool and Nis- sim (2016); Mohammad (2011), i.a.), we see two main shortcomings. First, there is shortage of re- sources for non-English languages, with few ex- ceptions, like Chinese (Li et al., 2017; Odbal and Wang, 2014; Yuan et al., 2002). This hampers the data-driven modeling of emotion recognition that has unfolded, e.g., for the related task of senti- ment analysis. Second, emotions can be expressed in language with a wide variety of linguistic de- vices, from direct mentions (e.g., “I’m angry”) to evocative images (e.g.,“He was petrified”) or prosody. Computational emotion recognition on English has mostly focused on explicit emotion expressions. Often, however, emotions are merely inferable from world knowledge and experience. For instance, ”I finally found love” presumably de- picts a joyful circumstance, while fear probably ensued when ”She heard a sinister sound”. Atten- tion to such event-related emotions is arguably im- portant for wide-coverage emotion recognition and has motivated shared tasks (Klinger et al., 2018), structured resources (Balahur et al., 2011) and ded- icated studies such as the “International Survey on Emotion Antecedents and Reactions” (ISEAR, Scherer and Wallbott, 1994). ISEAR, as one out- come, provides a corpus of English descriptions of emotional events for 7 emotions (anger, disgust, fear, guilt, joy, shame, sadness). Informants were asked in a classroom setting to describe emotional situations they experienced. This focus on private perspectives on events sets ISEAR apart. Even though from psychology, it is now established in natural language processing as a textual source of emotional events. With this paper, we publish and analyze deISEAR, a German corpus of emotional event de- scriptions, and its English companion enISEAR, each containing 1001 instances. We move be- yond the original …

Content truncated to 500 words