APPReddit: a Corpus of Reddit Posts Annotated for Appraisal

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3809–3818 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3809 APPReddit: a Corpus of Reddit Posts Annotated for Appraisal Marco Antonio Stranisci∗, Simona Frenda∗⋄, Eleonora Ceccaldi†, Valerio Basile∗, Rossana Damiano∗, Viviana Patti∗ ∗University of Turin - Departiment of Computer Science C.so Svizzera 185, 10149, Turin, Italy. ⋄Universitat Polit`ecnica de Val`encia - PRHLT Research Center, Valencia, Spain {marcoantonio.stranisci,simona.frenda,valerio.basile,rossana.damiano,viviana.patti}@unito.it †Casa Paganini - InfoMus, DIBRIS, University of Genoa P.zza S. Maria in Passione 34, 16123, Genoa, Italy. eleonoraceccaldi@gmail.com Abstract Despite the large number of computational resources for emotion recognition, there is a lack of data sets relying on appraisal models. According to Appraisal theories, emotions are the outcome of a multi-dimensional evaluation of events. In this paper, we present APPReddit, the first corpus of non-experimental data annotated according to this theory. After describing its development, we compare our resource with enISEAR, a corpus of events created in an experimental setting and annotated for appraisal. Results show that the two corpora can be mapped notwithstanding different typologies of data and annotations schemes. A SVM model trained on APPReddit predicts four appraisal dimensions without significant loss. Merging both corpora in a single training set increases the prediction of 3 out of 4 dimensions. Such findings pave the way to a better performing classification model for appraisal prediction. Keywords: Emotion Recognition, Appraisal Theories, Annotated Corpora 1. Introduction In the last 15 years, several corpora and lexicons have been developed with the aim of deepening the anal- ysis of sentiment in texts (Strapparava and Mihalcea, 2007; Warriner et al., 2013; Cambria et al., 2010). These resources are heterogeneous, reflecting the high variety of emotion theories in psychological litera- ture (see: Sander et al. (2018) for an extensive re- view). A first group (Cambria et al., 2010; Bertola and Patti, 2016; Li et al., 2017) relies on categorical emotion models, according to which there is a set of basic emotions emerged within the evolutionary pro- cess (Plutchik, 1991; Ekman, 1992). Another group of computational resources (Warriner et al., 2013; Buechel and Hahn, 2017; Mohammad, 2018) refers to dimensional models, which focus on three independent cognitive dimensions along which emotions are eval- uated and mapped: Valence, Arousal, and Dominance (Russell, 1980; Bradley and Lang, 2007). NLP resources modeled on appraisal theories, on the other hand, are still missing, with the exception of Hof- mann et al. (2020) and Scherer and Wallbott (1994). Theories of appraisal (Smith and Ellsworth, 1985; Roseman and Smith, 2001) emphasize the evaluation stage of an event or a situation, that leads to an emo- tional response and to a corresponding behavior aimed at coping with the situation and alleviating the response itself. Emotions, in this view, stem from cognitive eval- uations of events and are followed by specific auto- nomic responses, behavioral configurations and action tendencies (Smith and Ellsworth, 1985). Such evalu- ations work by assessing the current situation against a set of appraisal criteria, such as …

Content truncated to 500 words