From the Detection of Toxic Spans in Online Discussions to the Analysis of Toxic-to-Civil Transfer

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3721 - 3734 May 22-27, 2022 c⃝2022 Association for Computational Linguistics From the Detection of Toxic Spans in Online Discussions to the Analysis of Toxic-to-Civil Transfer John Pavlopoulos♠♣, Léo Laugier⋄, Alexandros Xenos♣, Jeffrey Sorensen‡, Ion Androutsopoulos♣ ♠Department of Computer and Systems Sciences, Stockholm University, Sweden ♣Department of Informatics, Athens University of Economics and Business, Greece ⋄ Télécom Paris, Institut Polytechnique de Paris, France ‡ Google {annis,a.xenos20,ion}@aueb.gr leo.laugier@telecom-paris.fr sorenj@google.com Abstract We study the task of toxic spans detection, which concerns the detection of the spans that make a text toxic, when detecting such spans is possible. We introduce a dataset for this task, TOXICSPANS, which we release publicly. By experimenting with several methods, we show that sequence labeling models perform best. Moreover, methods that add generic ratio- nale extraction mechanisms on top of classifiers trained to predict if a post is toxic or not are also surprisingly promising. Finally, we use TOXICSPANS and systems trained on it, to pro- vide further analysis of state-of-the-art toxic to non-toxic transfer systems, as well as of human performance on that latter task. Our work high- lights challenges in finer toxicity detection and mitigation. 1 Introduction In social media and online fora, toxic content can be defined as rude, disrespectful, or unreasonable posts that would make users want to leave the con- versation (Borkan et al., 2019). Although several toxicity detection datasets (Wulczyn et al., 2017; Borkan et al., 2019) and models (Schmidt and Wie- gand, 2017; Pavlopoulos et al., 2017c; Zampieri et al., 2019) exist, most of them classify whole posts, without identifying the specific spans that make a text toxic. But highlighting such toxic spans can assist human moderators (e.g., news portal moderators) who often deal with lengthy comments, and who prefer attribution instead of a system-generated unexplained toxicity score per post. Locating toxic spans within a text is thus a major step towards successful semi-automated moderation and healthier online discussions. To promote research on this new task, we release the first dataset of English posts with annotations of toxic spans, called TOXICSPANS.1 We discuss how it was created and propose an evaluation framework for toxic spans detection. We consider methods that (i) perform sequence labeling (tag words) or (ii) rely on an attentional binary classifier to predict if a post is toxic or not, then invoke its attention at inference time to obtain toxic spans as in ratio- nale extraction. The latter approach allows leverag- ing larger existing training datasets, which provide gold labels indicating which posts are toxic or not, without providing gold toxic span annotations. Al- though sequence labeling performed overall better, the binary attentional classifier performed surpris- ingly well too, despite having been trained on data without span annotations. We then study some characteristics of supervised and self-supervised toxic-to-civil transfer models (Laugier et al., 2021) by comparing them on sev- eral datasets, including a recently released parallel toxic-to-civil dataset (Dementieva et al., …

Content truncated to 500 words