An Emotional Mess! Deciding on a Framework for Building a Dutch Emotion-Annotated Corpus

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1643–1651 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1643 An Emotional Mess! Deciding on a Framework for Building a Dutch Emotion-Annotated Corpus Luna De Bruyne, Orph´ee De Clercq, Veronique Hoste LT3, Language and Translation Technology Team, Ghent University Groot-Brittanni¨elaan 45, 9000 Ghent, Belgium {luna.debruyne, orphee.declercq, veronique.hoste}@ugent.be Abstract Seeing the myriad of existing emotion models, with the categorical versus dimensional opposition the most important dividing line, building an emotion-annotated corpus requires some well thought-out strategies concerning framework choice. In our work on automatic emotion detection in Dutch texts, we investigate this problem by means of two case studies. We find that the labels joy, love, anger, sadness and fear are well-suited to annotate texts coming from various domains and topics, but that the connota- tion of the labels strongly depends on the origin of the texts. Moreover, it seems that information is lost when an emotional state is forcedly classified in a limited set of categories, indicating that a bi-representational format is desirable when creating an emotion corpus. Keywords: NLP, emotion detection, emotion annotation 1. Introduction When dealing with the task of automatically detecting emo- tions in texts – a well-studied topic in the field of natu- ral language processing or NLP (Mohammad et al., 2018; Chatterjee et al., 2019) – the first bottleneck is data acqui- sition. Not only is there the need to collect a considerable amount of data, one also needs to decide on an appropriate framework to annotate these data instances in order to build an emotion corpus. Seeing the plethora of existing emotion frameworks, this decision is not trivial. On the one hand, emotions can be represented as categories, typically by using a set of basic emotions. The frameworks of Ekman (1992) (with the basic emotions anger, disgust, fear, joy, sadness and surprise) and Plutchik (1980) (in which trust and anticipation are added) are the most pop- ular ones, but many other theorists provided sets of basic emotions, counting up to 14 emotion categories, e.g. the emotion theory of Roseman (1984). Dimensional models, on the other hand, represent emo- tions as vectors in a multidimensional space. Mehra- bian and Russell (1974) claimed that this emotional space is defined by the axes valence (unhappiness- happiness), arousal (calmness-excitement) and dominance (submission-dominance), so that every emotional state can be described as a combination of values on these VAD-axes. However, in later work, Russell (1980) argued that only the dimensions valence and arousal are necessary for describ- ing any emotional state, whereas Fontaine et al. (2007) claim a fourth dimension, unpredictability, is needed. Building an emotion-annotated corpus thus requires some well thought-out strategies concerning annotation method and framework choice. However, in most emotion datasets, the motives on which a particular emotion framework is se- lected are unclear or the choice even seems arbitrary. In almost all studies, preference is given to categorical ap- proaches for building datasets, but only very rarely …

Content truncated to 500 words