Placing M-Phasis on the Plurality of Hate: A Feature-Based Corpus of Hate Online

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 791–804 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 791 Placing M-Phasis on the Plurality of Hate: A Feature-Based Corpus of Hate Online Dana Ruiter†∗, Liane Reiners‡∗, Ashwin Geet D’Sa♣, Thomas Kleinbauer†, Dominique Fohr♣, Irina Illina♣, Dietrich Klakow†, Christian Schemer‡, Angeliki Monnier♠ †Spoken Language Systems Group, Saarland University, Germany {druiter, kleiba, dietrich.klakow}@lsv.uni-saarland.de ‡Department of Communication, Johannes Gutenberg University Mainz {liane.reiners, schemer}@uni-mainz.de ♣Universit´e de Lorraine, CNRS, Inria, LORIA {ashwin-geet.dsa, dominique.fohr, irina.illina}@loria.fr ♠ Universit´e de Lorraine, CREM angeliki.monnier@univ-lorraine.fr Abstract Even though hate speech (HS) online has been an important object of research in the last decade, most HS-related corpora over-simplify the phenomenon of hate by attempting to label user comments as hate or neutral. This ignores the complex and subjective nature of HS, which limits the real-life applicability of classifiers trained on these corpora. In this study, we present the M-Phasis corpus, a corpus of ∼ 9k German and French user comments collected from migration-related news articles. It goes beyond the hate-neutral dichotomy and is instead annotated with 23 features, which in combination become descriptors of various types of speech, ranging from critical comments to implicit and explicit expressions of hate. The annotations are performed by 4 native speakers per language and achieve high (0.77 ≤ κ ≤ 1) inter-annotator agreements. Besides describing the corpus creation and presenting insights from a content, error and domain analysis, we explore its data characteristics by training several classification baselines. Keywords: Hate Speech, Corpus Creation, Feature-Based, Multi-Disciplinary, Bilingual, Migration 1. Introduction The internet has made the exchange of information and ideas between individuals easier than ever before. But through the provision of anonymity and filter bubbles, it has also contributed to the propagation of hateful con- tents that are a threat to the open exchange of opin- ions. To gain insights into the dynamics and char- acteristics of different types of hateful speech and to develop counter-measures, communication researchers and computer scientists alike require high-quality an- notated data. To date, published datasets related to hate speech (HS) come with several limitations. Most corpora used to train HS classifiers over-simplify the phenomenon of HS by labelling user content with binary classes, e.g., hate/neutral, whose underlying definition varies greatly across corpora (Jurgens et al., 2019). Another common limitation in HS corpora is the use of slur (Rost et al., 2016) or emotion word lists (Paltoglou et al., 2013) to identify and sample hateful content. These approaches are insensitive to more subtle forms of HS (i.e., implicit hate) that is conveyed through syntactical or contextual features (Cohen-Almagor, 2018). Fur- ther, most hate speech corpora are based on the same ∗ Equal contribution. outlet, i.e., Twitter (more than 50% of datasets), and language, i.e., English (∼ 40%) (Vidgen and Derczyn- ski, 2021). Focusing on Twitter data is especially lim- iting, given that Twitter poses a special case of user in- teractions due to the small maximum length of tweets. The English-centrism ignores …

Content truncated to 500 words