Toxicity Detection: Does Context Really Matter?

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4296–4305 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4296 Toxicity Detection: Does Context Really Matter? John Pavlopoulos†, Jeffrey Sorensen‡ Lucas Dixon‡, Nithum Thain‡, Ion Androutsopoulos† † Department of Informatics, Athens University of Economic and Business, Greece annis,ion@aueb.gr ‡ Google sorenj,ldixon,nthain@google.com Abstract Moderation is crucial to promoting healthy on- line discussions. Although several ‘toxicity’ detection datasets and models have been pub- lished, most of them ignore the context of the posts, implicitly assuming that comments may be judged independently. We investigate this assumption by focusing on two questions: (a) does context affect the human judgement, and (b) does conditioning on context improve per- formance of toxicity detection systems? We experiment with Wikipedia conversations, lim- iting the notion of context to the previous post in the thread and the discussion title. We find that context can both amplify or mitigate the perceived toxicity of posts. Moreover, a small but significant subset of manually la- beled posts (5% in one of our experiments) end up having the opposite toxicity labels if the an- notators are not provided with context. Sur- prisingly, we also find no evidence that context actually improves the performance of toxicity classifiers, having tried a range of classifiers and mechanisms to make them context aware. This points to the need for larger datasets of comments annotated in context. We make our code and data publicly available. 1 Introduction Systems that detect abusive language are used to promote healthy conversations online and protect minority voices (Hosseini et al., 2017). Apart from a growing volume of press articles concerning toxi- city online,1 there is increased research interest on detecting abusive and other unwelcome comments labeled ‘toxic’ by moderators, both for English and other languages.2 However, the vast majority of 1Following the work of Wulczyn et al. (2017) and Borkan et al. (2019), toxicity is defined as “a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion” (Wulczyn et al., 2017). 2For English, see for example TRAC (Kumar et al., 2018), OFFENSEVAL (Zampieri et al., 2019b), or the recent Work- shops on Abusive Language Online (https://goo.gl/ PARENT All of his arguements are nail perfect, you’re inherently stupid. The lead will be changed. TARGET Great argument! PARENT Really? It’s schmucks like you (and Bush) who turn the world into the shithole it is today! TARGET I’d be interested in the reasoning for that comment, personally. (bounties) PARENT Indeed. Hitler was also strongly anti- pornography [...] it sure looks like Hitler is a hot potato that nobody wants to be stuck with. TARGET Well I guess they won’t approve the slogan “Hitler hated porn”. PARENT ?? When did I attack you? I definitely will present this to the arbcom, you should mind WP:CIVIL when participating in discussions in Wikipedia. TARGET I blame you for my alcoholism add that too Table 1: Comments that are not easily labeled for toxi- city without the ‘parent’ (previous) …

Content truncated to 500 words