RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1941–1955 August 1–6, 2021. ©2021 Association for Computational Linguistics 1941 REDDITBIAS: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models Soumya Barikeri,1 Anne Lauscher,1 Ivan Vuli´c,2 and Goran Glavaˇs1 1Data and Web Science Research Group University of Mannheim soumyabarikeri@gmail.com,{anne, goran}@informatik.uni-mannheim.de 2Language Technology Lab University of Cambridge iv250@cam.ac.uk Abstract Text representation models are prone to exhibit a range of societal biases, reflecting the non- controlled and biased nature of the underlying pretraining data, which consequently leads to severe ethical issues and even bias amplifica- tion. Recent work has predominantly focused on measuring and mitigating bias in pretrained language models. Surprisingly, the landscape of bias measurements and mitigation resources and methods for conversational language mod- els is still very scarce: it is limited to only a few types of bias, artificially constructed resources, and completely ignores the impact that debi- asing methods may have on the final perfor- mance in dialog tasks, e.g., conversational re- sponse generation. In this work, we present REDDITBIAS, the first conversational data set grounded in the actual human conversations from Reddit, allowing for bias measurement and mitigation across four important bias di- mensions: gender, race, religion, and queer- ness. Further, we develop an evaluation frame- work which simultaneously 1) measures bias on the developed REDDITBIAS resource, and 2) evaluates model capability in dialog tasks after model debiasing. We use the evaluation framework to benchmark the widely used con- versational DialoGPT model along with the adaptations of four debiasing methods. Our results indicate that DialoGPT is biased with respect to religious groups and that some de- biasing techniques can remove this bias while preserving downstream task performance. 1 Introduction Pretrained language models and their correspond- ing contextualized representation spaces (Peters et al., 2018; Devlin et al., 2019) have recently been shown to encode and amplify a range of stereo- typical human biases (e.g., gender or racial biases) (Zhao et al., 2019; Basta et al., 2019; Liang et al., 2020a,b), much like their static embedding pre- decessors (Bolukbasi et al., 2016; Caliskan et al., 2017; Dev and Phillips, 2019; Gonen and Gold- berg, 2019; Lauscher et al., 2020a, inter alia). Hav- ing models that capture or even amplify human biases brings about further ethical challenges to the society (Henderson et al., 2018), since stereotyp- ing minoritized groups is a representational harm that perpetuates societal inequalities and unfairness (Blodgett et al., 2020). Human biases are in all likelihood especially harmful if encoded in con- versational AI systems, like the recent DialoGPT model (Zhang et al., 2020), which directly interact with humans, possibly even taking part in intimate and personal conversations (Utami et al., 2017). Given the increasing presence of dialog systems and chatbots in everyday life, the body of work that focuses on detecting and mitigating biases in conversational systems is surprisingly limited (Lee et al., 2019; Liu et al., 2020a,b; Dinan et al., …

Content truncated to 500 words