ToKen: Task Decomposition and Knowledge Infusion for Few-Shot Hate Speech Detection

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2109 - 2120 December 7-11, 2022 ©2022 Association for Computational Linguistics TOKEN: Task Decomposition and Knowledge Infusion for Few-Shot Hate Speech Detection Badr AlKhamissi∗ Faisal Ladhak∗ Srini Iyer Ves Stoyanov Zornitsa Kozareva Xian Li Pascale Fung Lambert Mathias Asli Celikyilmaz Meta AI Mona Diab Abstract Hate speech detection is complex; it relies on commonsense reasoning, knowledge of stereo- types, and an understanding of social nuance that differs from one culture to the next. It is also difficult to collect a large-scale hate speech annotated dataset. In this work, we frame this problem as a few-shot learning task, and show significant gains with decomposing the task into its "constituent" parts. In addi- tion, we see that infusing knowledge from rea- soning datasets (e.g. ATOMIC20 20 ) improves the performance even further. Moreover, we observe that the trained models generalize to out-of-distribution datasets, showing the supe- riority of task decomposition and knowledge infusion compared to previously used methods. Concretely, our method outperforms the base- line by 17.83% absolute gain in the 16-shot case. 1 Introduction Disclaimer: Due to the nature of this work, some examples contain offensive text and hate speech. This does not reflect authors’ values, however our aim is to help detect and prevent the spread of such harmful content. The task of automatically detecting Hate Speech (HS) is becoming increasingly important given the rapid growth of social media platforms, and the severe social harms associated with the spread of hateful content. However, building good systems for automated HS detection is challenging due to the complex nature of the task. It requires the sys- tem to understand social nuance, such as which groups are being targeted by the hateful content. Prior work has shown that even humans cannot achieve a high agreement on whether or not a social media post constitutes HS (Rahman et al., 2021). In this work, we explore whether decompos- ing HS detection into subtasks that correspond ∗Equal Contribution Figure 1: Hate Speech decomposed into several sub- tasks leading to better results. to the definitional criteria of what constitutes HS (i.e. the offensiveness of a post and whether it tar- gets a group or an individual) (Davidson et al., 2017; Mollas et al., 2020a) would lead to sys- tems that are more accurate and robust. In par- ticular, we show that task decomposition leads to more sample-efficient systems for HS detection, by showing improved results in the few-shot setting. Moreover, we demonstrate that infusing common- sense knowledge by fine-tuning on the ATOMIC20 20 (Hwang et al., 2021) and StereoSet (Nadeem et al., 2021) datasets improve performance even further. Specifically, we observe an absolute improvement of 17.83% in the 16-shot case over the baseline model (§5.2). Further, we show that the resulting models are more robust, and are able to achieve better performance than baseline methods in out- of-distribution settings (§5.3). Explainability is an important aspect for being able to identify and fix failure modes …

Content truncated to 500 words