Explaining Toxic Text via Knowledge Enhanced Text Generation

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 811 - 826 July 10-15, 2022 ©2022 Association for Computational Linguistics Explaining Toxic Text via Knowledge Enhanced Text Generation Rohit Sridhar School of Interactive Computing Georgia Institute of Technology rsridhar37@gatech.edu Diyi Yang School of Interactive Computing Georgia Institute of Technology dyang888@gatech.edu Abstract Warning: This paper contains content that is offensive and may be upsetting. Biased or toxic speech can be harmful to var- ious demographic groups. Therefore, it is not only important for models to detect these speech, but to also output explanations of why a given text is toxic. Previous literature has mostly focused on classifying and detecting toxic speech, and existing efforts on explaining stereotypes in toxic speech mainly use standard text generation approaches, resulting in generic and repetitive explanations. Building on these prior works, we introduce a novel knowledge- informed encoder-decoder framework to utilize multiple knowledge sources to generate impli- cations of biased text. Experiments show that our knowledge informed models outperform prior state-of-the-art models significantly, and can generate detailed explanations of stereo- types in toxic speech compared to baselines, both quantitatively and qualitatively. 1 Introduction The toxic speech detection and classification prob- lem has seen increasing interest in recent years. However, it is not only important for AI agents to recognize and classify toxic speech, but to also explain why it is toxic. For instance, debiasing methods that use information about toxic language may benefit from additional information given by detailed explanations of toxicity in text (Ma et al., 2020). Furthermore, detailed explanations of tox- icity may facilitate human interaction with toxic- ity detection systems (Rosenfeld and Richardson, 2019). They can also help humans who work with toxicity classifiers use more information about the input when making decisions about toxic speech. To elucidate, consider the following offensive joke: “What type of punch do you use against a kinder- gartener? A sandy-hook.". While the literal text is not toxic, the implied meaning is offensive, partic- ularly to those affected by school shootings. An AI agent capable of generating the implied meaning could thus provide additional information to down- stream actors. Note that, we use the term biased and toxic interchangeably in this work. Existing work largely addresses the problem of detecting and classifying toxic speech (Waseem and Hovy, 2016; Founta et al., 2018; Davidson et al., 2017). As mentioned earlier, explanations of toxicity can help with downstream tasks such as debiasing or decision making by humans, thus there has been increasing demand for explainable machine learning classifiers (Ribeiro et al., 2016; Došilovi´c et al., 2018). Recent work around ex- plainable toxicity classification introduced Social Bias Frames (Sap et al., 2020), a formal framework which combines explanations of toxicity along with toxicity classifications along multiple dimensions. However the explanations generated from the cur- rent state-of-the-art methods tend to be generic, without much detail. For instance, explanations may focus on certain toxic components of the input but ignore others, …

Content truncated to 500 words