Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1896–1907 August 1–6, 2021. ©2021 Association for Computational Linguistics 1896 Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate Austin Botelho∗ Anti-Defamation League abotelho@adl.org Scott A. Hale University of Oxford and Meedan scott.hale@oii.ox.ac.uk Bertie Vidgen The Alan Turing Institute bvidgen@turing.ac.uk Abstract Accurate detection and classification of online hate is a difficult task. Implicit hate is particu- larly challenging as such content tends to have unusual syntax, polysemic words, and fewer markers of prejudice (e.g., slurs). This prob- lem is heightened with multimodal content, such as memes (combinations of text and im- ages), as they are often harder to decipher than unimodal content (e.g., text alone). This paper evaluates the role of semantic and multimodal context for detecting implicit and explicit hate. We show that both text- and visual- enrichment improves model performance, with the mul- timodal model (0.771) outperforming other models’ F1 scores (0.544, 0.737, and 0.754). While the unimodal-text context-aware (trans- former) model was the most accurate on the subtask of implicit hate detection, the multi- modal model outperformed it overall because of a lower propensity towards false positives. We find that all models perform better on con- tent with full annotator agreement and that multimodal models are best at classifying the content where annotators disagree. To con- duct these investigations, we undertook high- quality annotation of a sample of 5,000 mul- timodal entries. Tweets were annotated for primary category, modality, and strategy. We make this corpus, along with the codebook, code, and final model, freely available. 1 Introduction Although its prevalence is fairy low (Vidgen et al., 2019), the effects of online hate can be deeply per- nicious, risking real harm on targeted victims and their communities (M¨uller et al., 2019; Guadagno et al., 2013). A 2021 survey by Anti-Defamation League found that 81% of Americans agree social media companies should do more to counter online hate (Anti-Defamation League, 2021). ∗Work performed while at the University of Oxford Research into automated hate detection has pri- marily focused on explicit varieties. However, many purveyors of hate have adopted more com- plex and nuanced strategies, such as dog whistling: the use of intentionally ambiguous rhetorical tech- niques to express hateful messages which only some audiences will recognize. For instance, calls by right-wing American political figures to “protect the suburbs” cloak racial grievances and concerns about whiteness in more prosaic concerns about community protection. Performance in online hate classification has improved substantially from static methods like GloVe and fastText through the use of context- aware word embeddings, in particular those com- puted by transformer-models with self-attention (Badjatiya et al., 2019; Mozafari et al., 2020; Polig- nano et al., 2019; Sabat et al., 2019; Zampieri et al., 2019; Yang et al., 2019; Sohn and Lee, 2019; Kennedy et al., 2020; Vidgen et al., 2020). How- ever, most hate detection models are text-only and cannot be applied to non-textual content (such as images and audio) and do not account for non- textual …

Content truncated to 500 words