Challenges in Automated Debiasing for Toxic Language Detection

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3143–3155 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3143 Challenges in Automated Debiasing for Toxic Language Detection Xuhui Zhou♥ Maarten Sap♣ Swabha Swayamdipta♦ Noah A. Smith♣♦ Yejin Choi♣♦ ♥Department of Linguistics, University of Washington ♣Paul G. Allen School of Computer Science & Engineering, University of Washington ♦Allen Institute for Artificial Intelligence xuhuizh@uw.edu, msap@cs.washington.edu {swabhas,noah,yejinc}@allenai.org Abstract Warning: this paper contains content that may be offensive or upsetting. Biased associations have been a challenge in the development of classifiers for detecting toxic language, hindering both fairness and accuracy. As potential solutions, we inves- tigate recently introduced debiasing methods for text classification datasets and models, as applied to toxic language detection. Our focus is on lexical (e.g., swear words, slurs, iden- tity mentions) and dialectal markers (specifi- cally African American English). Our com- prehensive experiments establish that existing methods are limited in their ability to prevent biased behavior in current toxicity detectors. We then propose an automatic, dialect-aware data correction method, as a proof-of-concept study. Despite the use of synthetic labels, this method reduces dialectal associations with toxicity. Overall, our findings show that debi- asing a model trained on biased toxic language data is not as effective as simply relabeling the data to remove existing biases. 1 Introduction Current hate speech or toxic language detection1 systems exhibit problematic and discriminatory behavior that causes them to have disparate nega- tive impact on minority populations (Yasin, 2018; Guynn, 2020; Kim et al., 2020; Dias Oliva et al., 2020). Tweets simply containing a minority iden- tity mention are commonly flagged as toxic by cur- rent systems, in contrast to those containing ma- jority identity mentions, as illustrated in Figure 1. At the core of the issue are dataset biases, i.e., spurious correlations between surface patterns and annotated toxicity labels (§2), which stem from the data creation process (Sap et al., 2019). Pre- vious work has outlined two such biases for hate 1We use hate speech and toxic language interchangeably in this work, though their definitions do not perfectly align. Detected toxicity score I identify as a black gay woman. I identify as a straight white man. Fucking love this. Adolf Hilter is a great person. Identity bias (Lexical) Swear word bias (Lexical) What’s up, bro! Wussup, n*gga! Dialect/ Racial bias Pers. API Pers. API Pers. API Figure 1: Lexical items and dialect markers cause prob- lematic behavior for toxic language detection systems such as the widely used PerspectiveAPI. In the top two example pairs, statements with minority identity men- tions and swear words used inoffensively are flagged as toxic, but majority identity mentions or offensive state- ments without overt swearing are missed. The bottom pair shows dialect-based racial bias for two inoffensive greetings, where markers of African American English (AAE) trigger the toxicity detector. speech datasets (both shown in Figure 1): lexical bias which associates toxicity with the presence of certain words (e.g., profanities, identity mentions; Dixon …

Content truncated to 500 words