Data-Efficient Strategies for Expanding Hate Speech Detection into Under-Resourced Languages

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5674 - 5691 December 7-11, 2022 ©2022 Association for Computational Linguistics Data-Efficient Strategies for Expanding Hate Speech Detection into Under-Resourced Languages Paul Röttger1, Debora Nozza2, Federico Bianchi3, and Dirk Hovy2 1University of Oxford 2Bocconi University 3Stanford University Abstract Hate speech is a global phenomenon, but most hate speech datasets so far focus on English- language content. This hinders the develop- ment of more effective hate speech detection models in hundreds of languages spoken by bil- lions across the world. More data is needed, but annotating hateful content is expensive, time- consuming and potentially harmful to annota- tors. To mitigate these issues, we explore data- efficient strategies for expanding hate speech detection into under-resourced languages. In a series of experiments with mono- and mul- tilingual models across five non-English lan- guages, we find that 1) a small amount of target-language fine-tuning data is needed to achieve strong performance, 2) the benefits of using more such data decrease exponentially, and 3) initial fine-tuning on readily-available English data can partially substitute target- language data and improve model generalis- ability. Based on these findings, we formulate actionable recommendations for hate speech detection in low-resource language settings. Content warning: This article contains illus- trative examples of hateful language. 1 Introduction Hate speech is a global phenomenon, but most hate speech datasets so far focus on English-language content (Vidgen and Derczynski, 2020; Poletto et al., 2021). This hinders the development of effec- tive models for detecting hate speech in other lan- guages. As a consequence, billions of non-English speakers across the world are less protected against online hate, and even giant social media platforms have clear language gaps in their content modera- tion systems (Simonite, 2021; Marinescu, 2021). Zero-shot cross-lingual transfer, where large multilingual language models are fine-tuned on one source language and then applied to another target language, may appear like a potential so- lution to the issue of language-specific resource Figure 1: Overview of our experimental setup. We use ISO 639-1 codes to denote the different languages. MHC is Multilingual HateCheck (Röttger et al., 2022). scarcity. However, while this approach performs well on some tasks (Conneau et al., 2020; Barbieri et al., 2022), it fails on many others (Lauscher et al., 2020; Hu et al., 2020). For hate speech detection in particular, cross-lingual performance in zero- shot settings is lacking (Stappen et al., 2020; Leite et al., 2020). For example, zero-shot cross-lingual transfer cannot account for language-specific taboo expressions that play a key role in classification (Nozza, 2021). Conversely, hate speech detection models trained or fine-tuned directly on the target language, i.e. in few- and many-shot settings, are consistently found to perform best (Aluru et al., 2020; Pelicon et al., 2021). So, how do we build hate speech detection models for hundreds more languages? We need at least some labelled data in the target language to make models effective, but data annotation is diffi- cult, time-consuming and expensive. It requires re- …

Content truncated to 500 words