Transfer Learning and Prediction Consistency for Detecting Offensive Spans of Text

Findings of the Association for Computational Linguistics: ACL 2022, pages 1630 - 1637 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Transfer Learning and Prediction Consistency for Detecting Offensive Spans of Text Amir Pouran Ben Veyseh1, Ning Xu2, Quan Hung Tran2, Varun Manjunatha2, Franck Dernoncourt2, Thien Huu Nguyen1 1Department of Computer and Information Science, University of Oregon, Eugene, OR, USA 2Adobe Research, San Jose, CA, USA {apouranb,thien}@cs.uoregon.edu {nxu,qtran,vmanjuna,franck.dernoncourt}@adobe.com Abstract Toxic span detection is the task of recogniz- ing offensive spans in a text snippet. Al- though there has been prior work on classify- ing text snippets as offensive or not, the task of recognizing spans responsible for the tox- icity of a text is not explored yet. In this work, we introduce a novel multi-task frame- work for toxic span detection in which the model seeks to simultaneously predict offen- sive words and opinion phrases to leverage their inter-dependencies and improve the per- formance. Moreover, we introduce a novel reg- ularization mechanism to encourage the con- sistency of the model predictions across sim- ilar inputs for toxic span detection. Our ex- tensive experiments demonstrate the effective- ness of the proposed model compared to strong baselines. 1 Introduction With the proliferation of social networks, the amount of textual data posted online is also ever- increasing. This growth comes with some chal- lenges too. One of the issues associated with social networks is the level of toxicity expressed in posts or comments shared online. The toxic/offensive languages in social networks can be realized in different forms such as insults, mockeries, threats, discrimination, or swearing. Due to their detrimen- tal effect on users of social networks, it is desirable to identify and remove offensive text from these networks. Since this is an important requirement, the task of offensive language detection has been exten- sively studied in NLP community (Schmidt and Wiegand, 2017; Feng et al., 2018; Borkan et al., 2019; Sivanaiah et al., 2020; Yasaswini et al., 2021). However, most of the existing works are limited to classifying a text snippet as offensive or not. In other words, these models fail to provide fur- ther information about what specific phrases in the text snippet contribute the most to the offensive tone of the text. This information is necessary for the moderators to decide further actions for the posts/comments flagged as offensive, especially if the text snippet is long. As such, in this work, we fill this gap by proposing a novel model for the task of offensive span detection (OSD). As an example, in the given text “This livestreamer clearly has no brain; he is such a tool!", the phrase “has no brain" and the slang word “tool" are two offensive spans responsible for the toxicity of the text. One of the barriers for this task is data scarcity. To address this limitation, we propose a novel model trained in multi-task setting in which the model is trained on two tasks: (1) Offensive phrase detection whose goal is to detect word(s) contributing to …

Content truncated to 500 words