How Hate Speech Varies by Target Identity: A Computational Analysis

How Hate Speech Varies by Target Identity: A Computational Analysis Michael Miller Yoder1 Lynnette Hui Xian Ng1 David West Brown2 Kathleen M. Carley1 1School of Computer Science 2Department of English Carnegie Mellon University Pittsburgh, Pennsylvania, USA {mamille3,huixiann,dwb2,carley}@andrew.cmu.edu Abstract This paper investigates how hate speech varies in systematic ways according to the identi- ties it targets. Across multiple hate speech datasets annotated for targeted identities, we find that classifiers trained on hate speech tar- geting specific identity groups struggle to gen- eralize to other targeted identities. This pro- vides empirical evidence for differences in hate speech by target identity; we then investigate which patterns structure this variation. We find that the targeted demographic category (e.g. gender/sexuality or race/ethnicity) appears to have a greater effect on the language of hate speech than does the relative social power of the targeted identity group. We also find that words associated with hate speech targeting spe- cific identities often relate to stereotypes, his- tories of oppression, current social movements, and other social contexts specific to identities. These experiments suggest the importance of considering targeted identity, as well as the so- cial contexts associated with these identities, in automated hate speech classification. Warning: This paper contains offensive and hateful terms and concepts. We have chosen to reproduce these terms for clarity in aiding efforts against hate speech. 1 Introduction Researchers working in natural language process- ing (NLP) often treat hate speech as a binary, uni- fied, concept that can be detected from language alone. However, as a linguistic concept that relies heavily on social context, hate speech contains a variety of related phenomena (Brown, 2017). Hate speech is characterized by variation in linguistic features (e.g. implicit vs. explicit), context (e.g. platforms, prior conversations), and communities (social histories and hierarchies). This paper fo- cuses on a crucial aspect of this variation: how hate speech varies by the identity groups it targets. To study this variation, we analyze hate speech datasets that include annotations for which identity group is targeted. Drawing from multiple of these datasets, we sample new corpora that target the same identity group. These identity groups vary according to several dimensions, including relevant demographic category (e.g. gender, religion) and relative social power (e.g. socially marginalized or dominant). We empirically test which dimensions most clearly separate different forms of hate speech by evaluating how well classifiers trained on one set of identities generalize to hate speech directed at different sets of identities. We find that hate speech varies most prominently by the targeted demographic category and less so by the social power of the targeted identity group. Theorists working in philosophy and sociolinguis- tics have drawn attention to how hate speech di- rected at marginalized groups differs from hate directed toward socially dominant groups (Butler, 1997; Lakoff, 2000). However, we do not find that hate speech toward dominant groups is sufficiently different to consistently increase classification per- formance when removed from existing datasets. Analyzing the most representative terms in hate speech directed toward different identities, we …

Content truncated to 500 words