Rethinking Offensive Text Detection as a Multi-Hop Reasoning Problem

Findings of the Association for Computational Linguistics: ACL 2022, pages 3888 - 3905 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Rethinking Offensive Text Detection as a Multi-Hop Reasoning Problem Qiang Zhang, Jason Naradowsky, Yusuke Miyao Department of Computer Science, The University of Tokyo, Tokyo {qiang-z714, narad}@g.ecc.u-tokyo.ac.jp yusuke@is.s.u-tokyo.ac.jp Abstract We introduce the task of implicit offensive text detection in dialogues, where a statement may have either an offensive or non-offensive inter- pretation, depending on the listener and context. We argue that reasoning is crucial for under- standing this broader class of offensive utter- ances and release SLIGHT, a dataset to support research on this task. Experiments using the data show that state-of-the-art methods of of- fense detection perform poorly when asked to detect implicitly offensive statements, achiev- ing only ∼11% accuracy. In contrast to existing offensive text detection datasets, SLIGHT features human-annotated chains of reasoning which describe the mental process by which an offensive interpretation can be reached from each ambiguous statement. We explore the potential for a multi-hop reason- ing approach by utilizing existing entailment models to score the probability of these chains and show that even naive reasoning models can yield improved performance in most situations. Furthermore, analysis of the chains provides insight into the human interpretation process and emphasizes the importance of incorporat- ing additional commonsense knowledge. 1 Introduction With the development and popularity of online fo- rums and social media platforms, the world is be- coming an increasingly connected place to share information and opinions. However, the benefit that these platforms provide to society is often marred by the creation of an unprecedented amount of bullying, hate, and other abusive speech1. Such toxic speech has detrimental effects on online com- munities and can cause significant personal harm. Efforts by the NLP community to address this prob- lem has led to the development of models capable 1Disclaimer: due to the nature of this work, data and ex- amples may contain content which is offensive to the reader. of identifying toxic speech in specific domains (sex- ism (Golbeck et al., 2017), racism (Waseem, 2016), or otherwise hateful text (Ross et al., 2016; Gao and Huang, 2017; Davidson et al., 2017)), but the prob- lem of identifying harmful text can also involve more complex pragmatic reasoning. Consider a scenario where a young girl runs into her elderly neighbor who remarks, “Your piano playing has really improved lately!” Most people (and classifiers) would likely take this comment as a compliment. However, in some circumstances, the intent may be the opposite. The neighbor can only have knowledge of the girl’s piano progress if she is able to hear it, and being able to hear it may indicate that it is too loud, implying that the girl is inconsiderate of her neighbors.2 Through this reasoning process, we may reach the less compli- mentary interpretation, namely that the neighbor is annoyed by the playing and the comment is a subtle attempt to convey it. This work considers how current models of offensive text …

Content truncated to 500 words