Token Sequence Labeling vs. Clause Classification for English Emotion Stimulus Detection

Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics (*SEM), pages 58–70 Barcelona, Spain (Online), December 12–13, 2020 58 Token Sequence Labeling vs. Clause Classification for English Emotion Stimulus Detection Laura Oberl¨ander and Roman Klinger Institut f¨ur Maschinelle Sprachverarbeitung, University of Stuttgart Pfaffenwaldring 5b, 70569 Stuttgart, Germany {laura.oberlaender,roman.klinger}@ims.uni-stuttgart.de Abstract Emotion stimulus detection is the task of find- ing the cause of an emotion in a textual descrip- tion, similar to target or aspect detection for sentiment analysis. Previous work approached this in three ways, namely (1) as text classifi- cation into an inventory of predefined possible stimuli (“Is the stimulus category A or B?”), (2) as sequence labeling of tokens (“Which tokens describe the stimulus?”), and (3) as clause clas- sification (“Does this clause contain the emo- tion stimulus?”). So far, setting (3) has been evaluated broadly on Mandarin and (2) on En- glish, but no comparison has been performed. Therefore, we analyze whether clause classi- fication or token sequence labeling is better suited for emotion stimulus detection in En- glish. We propose an integrated framework which enables us to evaluate the two different approaches comparably, implement models in- spired by state-of-the-art approaches in Man- darin, and test them on four English data sets from different domains. Our results show that token sequence labeling is superior on three out of four datasets, in both clause-based and token sequence-based evaluation. The only case in which clause classification performs better is one data set with a high density of clause annotations. Our error analysis further confirms quantitatively and qualitatively that clauses are not the appropriate stimulus unit in English. 1 Introduction Research in emotion analysis from text focuses on classification, i.e., mapping sentences or docu- ments to emotion categories based on psychologi- cal theories (e.g., Ekman (1992), Plutchik (2001)). While this task answers the question which emotion This work is licensed under a Creative Commons Attri- bution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/. is expressed in a text, it does not detect the textual unit, which reveals why the emotion has been devel- oped. For instance, in the example “Paul is angry because he lost his wallet.” it remains hidden that lost his wallet is the reason for experiencing the emotion of anger. This stimulus, e.g., an event de- scription, a person, a state of affairs, or an object enables deeper insight, similar to targeted or aspect- based sentiment analysis (Jakob and Gurevych, 2010; Yang and Cardie, 2013; Klinger and Cimiano, 2013; Pontiki et al., 2015, 2016, i.a.). This situa- tion is dissatisfying for (at least) two reasons. First, detecting the emotions expressed in social media and their stimuli might play a role in understanding why different social groups change their attitude towards specific events and could help recognize specific issues in society. Second, understanding the relationship between stimuli and emotions is also compelling from a psychological point of view, given that emotions are commonly considered re- sponses to relevant situations (Scherer, 2005). Models which tackle the task of …

Content truncated to 500 words