Detecting Unintended Social Bias in Toxic Language Datasets

Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 132 - 143 December 7-8, 2022 ©2022 Association for Computational Linguistics Detecting Unintended Social Bias in Toxic Language Datasets Nihar Sahoo∗, Himanshu Gupta∗, Pushpak Bhattacharyya CFILT, Indian Institute of Technology Bombay, India {nihar, himanshug, pb @cse.iitb.ac.in} Abstract Warning: This paper has contents which may be offensive, or upsetting however this cannot be avoided owing to the nature of the work. With the rise of online hate speech, automatic detection of Hate Speech, Offensive texts as a natural language processing task is getting popular. However, very little research has been done to detect unintended social bias from these toxic language datasets. This paper introduces a new dataset ToxicBias curated from the ex- isting dataset of Kaggle competition named "Jigsaw Unintended Bias in Toxicity Classifica- tion". We aim to detect social biases, their cat- egories, and targeted groups. The dataset con- tains instances annotated for five different bias categories, viz., gender, race/ethnicity, religion, political, and LGBTQ. We train transformer- based models using our curated datasets and report baseline performance for bias identifica- tion, target generation, and bias implications. Model biases and their mitigation are also dis- cussed in detail. Our study motivates a system- atic extraction of social bias data from toxic language datasets. All the codes and dataset used for experiments in this work are publicly available1. 1 Introduction In the age of social media and communications, it is simpler than ever to openly express one’s opinions on a wide range of issues. This openness results in a flood of useful information that can assist people in being more productive and making better decisions. According to statista 2, the global number of active social media users has just surpassed four billion, accounting for more than half of the world’s popu- lation. The user base is expected to grow steadily over the next five years. Various studies (Plaisime *These authors contributed equally to this work 1https://github.com/sahoonihar/ToxicBias_ CoNLL_2022 2https://www.statista.com/statistics/278414/ number-of-worldwide-social-network-users/ Figure 1: An illustrative example of ToxicBias. During the annotation process, hate speech/offensive text is provided without context. Annotators are asked to mark it as biased/neutral and to provide category, target, and implication if it has biases. et al., 2020) say that children and teenagers, who are susceptible, make up a big share of social me- dia users. Unfortunately, this increasing number of social media users also leads to an increase in toxicity (Matamoros-Fernández and Farkas, 2021). Sometimes this toxicity gives birth to violence and hate crimes. It does not just harm an individual; most of the time, the entire community suffers as due to its intensity. We have different perspectives based on race, gender, religion, sexual orientation, and many other factors. These perspectives sometimes lead to bi- ases that influence how we see the world, even if we are unaware of them. Biases like this can lead us to make decisions that are neither intelligent nor just. Furthermore, when these biases are expressed as hate speech and offensive texts, it …

Content truncated to 500 words