Dataset and Baseline for Automatic Student Feedback Analysis

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 193–201 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 193 Angry or Sad ? Emotion Annotation for Extremist Content Characterization V. Dragos, D. Battistelli, A. Etienne, Y. Constable ONERA - The French Aerospace Lab, France University of Paris Nanterre, France valentina.dragos@onera.fr, delphine.battistelli@parisnanterre.fr, aline.etienne@parisnanterre.fr, yolene.constable@onera.fr Abstract This paper examines the role of emotion annotations to characterize extremist content released on social platforms. The analysis of extremist content is important to identify user emotions towards some extremist ideas and to highlight the root cause of where emotions and extremist attitudes merge together. To address these issues our methodology combines knowledge from sociological and linguistic annotations to explore French extremist content collected online. For emotion linguistic analysis, the solution presented in this paper relies on a complex linguistic annotation scheme. The scheme was used to annotate extremist text corpora in French. Data sets were collected online by following semi-automatic procedures for content selection and validation. The paper describes the integrated annotation scheme, the annotation protocol that was set-up for French corpora annotation and the results, e.g. agreement measures and remarks on annotation disagreements. The aim of this work is twofold: first, to provide a characterization of extremist contents; second, to validate the annotation scheme and to test its capacity to capture and describe various aspects of emotions. Keywords: Emotion annotation, social data analysis, extremist content 1. Introduction Social platforms play an increasingly important role in the propagation of extremist ideas. As the growth of extremism online continues, the ability to detect this harmful content is paramount to restrain the spread of messages. Emotions associated to this type of content reflect the affiliations and aptitudes of users towards entities, events but also ideas and give clues about their activities online. A major barrier to the development of accurate models for extremist content detection is the limited number or rele- vant corpora and the absence of a reliable protocol for data annotation. More specifically, the need for corpora anno- tated by combining both sociological knowledge as to why the content is extremist has also emerged. The paper focuses on extremism analysis in French textual contents collected on social platforms and adopts an emo- tion annotation scheme to investigate emotions expressed in extremist and non-extremist bodies of texts. In this work, we show that extremist contents capture valuable emotional linguistic signals revealing, for instance, when extremist at- titudes are associated with fear or sadness. Building a labeled corpus for extremism detection is not trivial, since there is no consensus among researchers in the field of sociology on the definition of extremist content or the identification of its main characteristics (Alava et al., 2020). In addition, detecting emotion in text is a difficult task even for humans ( ¨Ohman, 2020). In order to cope with those difficulties, we developed an approach which con- sists of two main phases: (1) the development of a semi- automatic procedure guided by knowledge …

Content truncated to 500 words