SOS: Systematic Offensive Stereotyping Bias in Word Embeddings

Proceedings of the 29th International Conference on Computational Linguistics, pages 1263–1274 October 12–17, 2022. 1263 SOS: Systematic Offensive Stereotyping Bias in Word Embeddings Fatma Elsafoury1, Steven R. Wilson2, Stamos Katsigiannis3, and Naeem Ramzan1 1School of Physics, Engineering, and Computing, the University of The West of Scotland, UK 2Department of Computer Science and Engineering, Oakland University, USA 3Department of Computer Science, Durham University, UK Abstract Systematic Offensive stereotyping (SOS) in word embeddings could lead to associating marginalised groups with hate speech and pro- fanity, which might lead to blocking and silenc- ing those groups, especially on social media platforms. In this work, we introduce a quan- titative measure of the SOS bias, validate it in the most commonly used word embeddings, and investigate if it explains the performance of different word embeddings on the task of hate speech detection. Results show that SOS bias exists in almost all examined word embed- dings and that the proposed SOS bias metric correlates positively with the statistics of pub- lished surveys on online extremism. We also show that the proposed metric reveals distinct information compared to established social bias metrics. However, we do not find evidence that SOS bias explains the performance of hate speech detection models based on the different word embeddings. 1 Introduction Wagner et al. (2021) describe algorithmically in- fused societies as the societies that are shaped by algorithmic and human behaviour. The data col- lected from these societies carry the same bias in algorithms and humans, like population bias and behavioural bias (Olteanu et al., 2019). These bi- ases are important in the field of natural language processing (NLP) because unsupervised models like word embeddings encode them during training (Brunet et al., 2019; Joseph and Morgan, 2020). This includes racial bias which measures stereo- types related to people from different races, e.g. “Asians are good at math” (Garg et al., 2018; Manzini et al., 2019; Sweeney and Najafian, 2019), and gender bias which measures gender stereo- types, e.g. “women are housewives” (Garg et al., 2018; Bolukbasi et al., 2016; Chaloner and Maldon- ado, 2019). However, one aspect of bias that has received less attention is offensive stereotyping to- ward marginalised groups. For example, using slurs to describe non-white or LGBTQ communities or using swear words to describe women. Recent social research shows that using racial slurs and third-person profanity to describe groups of people aims at stressing the inferiority of the identity of the marginalised group (Kukla, 2018). Hence, as the internet is rife with slurs and profanity, it is important to study how machine learning models encode this offensive stereotyping. To this end, we extend our initial work on intro- ducing a computational measure of systematic of- fensive stereotyping (SOS) bias and examine its ex- istence in pre-trained word embeddings (Elsafoury, 2022). We define SOS from a statistical perspective as “A systematic association in the word embed- dings between profanity and marginalised groups of people”. In other words, SOS refers to associat- ing slurs and profane terms with different …

Content truncated to 500 words