“It’s Not Just Hate”: A Multi-Dimensional Perspective on Detecting Harmful Speech Online

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8093 - 8099 December 7-11, 2022 ©2022 Association for Computational Linguistics “It’s Not Just Hate”: A Multi-Dimensional Perspective on Detecting Harmful Speech Online Federico Bianchi Stanford University Stanford, California, USA Stefanie Anja Hills University of Stirling Stirling, UK Patricia Rossini University of Glasgow Glasgow, UK Dirk Hovy Bocconi University Milan, Italy Rebekah Tromble George Washington University Washington, DC, USA Nava Tintarev Maastricht University Maastricht, The Netherlands Abstract Well-annotated data is a prerequisite for good Natural Language Processing models. Too of- ten, though, annotation decisions are governed by optimizing time or annotator agreement. We make a case for nuanced efforts in an interdisci- plinary setting for annotating offensive online speech. Detecting offensive content is rapidly becoming one of the most important real-world NLP tasks. However, most datasets use a single binary label, e.g., for hate or incivility, even though each concept is multi-faceted. This modeling choice severely limits nuanced in- sights, but also performance. We show that a more fine-grained multi-label approach to predicting incivility and hateful or intolerant content addresses both conceptual and perfor- mance issues. We release a novel dataset of over 40,000 tweets about immigration from the US and UK, annotated with six labels for dif- ferent aspects of incivility and intolerance. Our dataset not only allows for a more nuanced un- derstanding of harmful speech online, models trained on it also outperform or match perfor- mance on benchmark datasets. Warning: This paper contains examples of hateful language some readers might find of- fensive. 1 Introduction Though once considered a problem driven primar- ily by reduced inhibitions in anonymous online spaces (Rösner and Krämer, 2016; Suler, 2004), offensive content has grown exponentially–to the point that many users no longer feel restricted by traditional conversational norms of tolerance and politeness, even when posting in their own names (Rossini, 2022). The pervasiveness of toxic discourse on social media in particular has helped sow the seeds of discord and hatred that harm the health and well-being of its targets and pose signifi- cant threats to the fundamental rights of individuals and social groups on the margins (Gelber and Mc- Namara, 2016). Concerned about such outcomes, over the last two decades scholars and practitioners from a variety of fields have scrutinized online inci- vility and hateful speech. Those working in natural language processing, for example, have developed techniques to detect different types of offensive discourse, ranging from incivility to hate speech, while social scientists have focused extensively on the larger substantive effects of these phenomena. Most computational approaches for detecting on- line toxicity are based on classifiers that predict the presence of a single main binary label (Basile et al., 2019; Stoll et al., 2020; Davidson et al., 2020, 2017), with some notable exceptions (Vid- gen et al., 2021a,b; Mollas et al., 2022; Kennedy et al., 2022, inter alia).1 However, while single- label binary approaches to harmful speech detec- tion are conceptually tidy and tend to yield good predictive …

Content truncated to 500 words