Natural Language Inference Prompts for Zero-shot Emotion Classification in Text across Corpora

Proceedings of the 29th International Conference on Computational Linguistics, pages 6805–6817 October 12–17, 2022. 6805 Natural Language Inference Prompts for Zero-shot Emotion Classification in Text across Corpora Flor Miriam Plaza-del-Arco1,2, María-Teresa Martín-Valdivia1, Roman Klinger2 1SINAI, Computer Science Department, CEATIC, Universidad de Jaén, Spain 2Institut für Maschinelle Sprachverarbeitung, University of Stuttgart, Germany {fmplaza, maite}@ujaen.es, klinger@ims.uni-stuttgart.de Abstract Within textual emotion classification, the set of relevant labels depends on the domain and application scenario and might not be known at the time of model development. This conflicts with the classical paradigm of supervised learn- ing in which the labels need to be predefined. A solution to obtain a model with a flexible set of labels is to use the paradigm of zero-shot learn- ing as a natural language inference task, which in addition adds the advantage of not needing any labeled training data. This raises the ques- tion how to prompt a natural language inference model for zero-shot learning emotion classifica- tion. Options for prompt formulations include the emotion name anger alone or the statement “This text expresses anger”. With this paper, we analyze how sensitive a natural language inference-based zero-shot-learning classifier is to such changes to the prompt under consid- eration of the corpus: How carefully does the prompt need to be selected? We perform experi- ments on an established set of emotion datasets presenting different language registers accord- ing to different sources (tweets, events, blogs) with three natural language inference models and show that indeed the choice of a particu- lar prompt formulation needs to fit to the cor- pus. We show that this challenge can be tackled with combinations of multiple prompts. Such ensemble is more robust across corpora than individual prompts and shows nearly the same performance as the individual best prompt for a particular corpus. 1 Introduction To enable communication about emotions, there exists a set of various emotion names, for instance those labeled as basic emotions, by Ekman (1992) or Plutchik (2001) (anger, fear, joy, sadness, dis- gust, surprise, trust, anticipation). While such psy- chological models influence natural language pro- cessing and emotion categorization approaches, the choice of emotion concepts is context-dependent. For instance, Scherer and Wallbott (1997) and Troiano et al. (2019) opted to use guilt and shame as self-directed emotions in addition to Ekman’s basic emotions, to analyze self-reports of events. For the context of the perception of art it is more appropriate to consider aesthetic emotions (Men- ninghaus et al., 2019; Haider et al., 2020), like beauty, sublime, inspiration, nostalgia, and melan- cholia. This leads to a potential gap between concepts in emotion-related training data and the applica- tion domain, purely because the label set is not compatible. One solution is to resort to so-called dimensional models, in which emotion names are located in vector spaces of affect (valence, arousal, Preo¸tiuc-Pietro et al., 2016; Buechel and Hahn, 2017) or cognitive appraisal (e.g., regarding respon- sibility, certainty, pleasantness, control, attention with respect to a stimulus event, Hofmann et al., 2020; Troiano et al., 2023). In …

Content truncated to 500 words