Explainable Abuse Detection as Intent Classification and Slot Filling

Explainable Abuse Detection as Intent Classification and Slot Filling Agostina Calabrese and Bj¨orn Ross and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB, United Kingdom {a.calabrese@,b.ross@,mlap@inf.}ed.ac.uk Abstract To proactively offer social media users a safe online experience, there is a need for systems that can detect harmful posts and promptly alert platform moderators. In order to guar- antee the enforcement of a consistent policy, moderators are provided with detailed guide- lines. In contrast, most state-of-the-art models learn what abuse is from labeled examples and as a result base their predictions on spurious cues, such as the presence of group identi- fiers, which can be unreliable. In this work we introduce the concept of policy-aware abuse detection, abandoning the unrealistic expec- tation that systems can reliably learn which phenomena constitute abuse from inspecting the data alone. We propose a machine-friendly representation of the policy that moderators wish to enforce, by breaking it down into a collection of intents and slots. We collect and annotate a dataset of 3,535 English posts with such slots, and show how architectures for in- tent classification and slot filling can be used for abuse detection, while providing a ratio- nale for model decisions.1 1 Introduction The central goal of online content moderation is to offer users a safer experience by taking actions against abusive behaviors, such as hate speech. Researchers have been developing super- vised classifiers to detect hateful content, starting from a collection of posts known to be abusive and non-abusive. To successfully accomplish this task, models are expected to learn complex concepts from previously flagged examples. For example, hate speech has been defined as ‘‘abusive speech targeting specific group characteristics, such as ethnic origin, religion, gender or sexual orienta- 1Our code and data are available at https://github .com/Ago3/PLEAD. tion’’ (Warner and Hirschberg, 2012), but there is no clear definition of what constitutes abusive speech. Recent research (Dixon et al., 2018) has shown that supervised models fail to grasp these complex- ities; instead, they exploit spurious correlations in the data, they become overly reliant on low-level lexical features and flag posts because of, for instance, the presence of group identifiers alone (e.g., women or gay). Efforts to mitigate these problems focus on regularization, for example, preventing the model from paying attention to group identifiers during training (Kennedy et al., 2020; Zhang et al., 2020), however, they do not seem effective at producing better classifiers (Calabrese et al., 2021). Social media companies, on the other hand, give moderators detailed guide- lines to help them decide whether a post should be deleted, and these guidelines also help en- sure consistency in their decisions (see Table 1). Models are not given access to these guidelines, and arguably this is the reason for many of their documented weaknesses. Let us illustrate this with the following exam- ple. Assume we are shown two posts, the abusive ‘‘Immigrants are parasites’’, and the non-abusive ‘‘I love artists’’, and are asked …

Content truncated to 500 words