Socially Responsible Hate Speech Detection: Can Classifiers Reflect Social Stereotypes?

Proceedings of Recent Advances in Natural Language Processing, pages 1187–1196 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_126 1187 Socially Responsible Hate Speech Detection: Can Classifiers Reflect Social Stereotypes? Francielle Vargas1,2, Isabelle Carvalho1, Ali H¨urriyeto˘glu3 Thiago A. S. Pardo1, Fabr´ıcio Benevenuto2 1Institute of Mathematical and Computer Sciences, University of S˜ao Paulo, Brazil 2Computer Science Department, Federal University of Minas Gerais, Brazil 3 KNAW Humanities Cluster, DHLab, the Netherlands francielleavargas@usp.br Abstract Recent studies have shown that hate speech technologies may propagate social stereotypes against marginalized groups. Nevertheless, there has been a lack of realistic approaches to assess and mitigate biased technologies. In this paper, we introduce a new approach to ana- lyze the potential of hate-speech classifiers to reflect social stereotypes through the investi- gation of stereotypical beliefs by contrasting them with counter-stereotypes. We empirically measure the distribution of stereotypical be- liefs by analyzing the distinctive classification of tuples containing stereotypes versus counter- stereotypes in machine learning models and datasets. Experiment results show that hate speech classifiers attribute unreal or negligent offensiveness to social identity groups by re- flecting and reinforcing stereotypical beliefs regarding minorities. Furthermore, we also found out that models that embed expert and context information from offensiveness mark- ers present promising results to mitigate social stereotype bias towards socially responsible hate speech detection.1 1 Introduction While Artificial Intelligence (AI) technologies have generated unprecedented opportunities for society, they have also introduced new forms of perpetu- ating inequality and heightened threats to human rights and well-being (UN, 2023). In this context, the investigation of Hate Speech (HS) is undoubtedly important since the propo- sition of automated systems has implications for unprejudiced societies. Nevertheless, researchers have constantly observed that these technologies are being developed with scarce consideration of their potential social biases, which may perpetu- ate social inequalities when propagated at scale 1Warning: This paper contains examples of offensive content and stereotypes. It does not reflect our way of thinking. Examples Classes They’ve been known to disregard people who march in Gay Pride parades. Hate Speech They’ve been known to disregard people who march in Christmas parades. Non-Hate Speech They’ve been known to disregard people who march in parades. Non-Hate Speech We can’t go to that one in this Mexican neighborhood. You might be robbed. Non-Hate Speech We can’t go to that one in this Canadian neighborhood. You might be robbed. Hate Speech We can’t go to that one in this neighbor- hood. You might be robbed. Hate Speech Table 1: Stereotype Bias in Hate Speech Detection. (Davani et al., 2023; Blodgett et al., 2020; Chuang et al., 2021; Xia et al., 2020; Wiegand et al., 2019; Sap et al., 2019; Bordia and Bowman, 2019; David- son et al., 2019). For example, Table 1 shows that the hate speech classifier attributed unreal offen- siveness to the first example only due to the ex- pression “Gay Pride”, which represents a social identity2 group. We observe that in the second ex- ample, the expression “Gay Pride” was replaced by “Christmas”, and in the third example, they …

Content truncated to 500 words