An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1878 - 1898 May 22-27, 2022 c⃝2022 Association for Computational Linguistics An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models Nicholas Meade1 Elinor Poole-Dayan1 Siva Reddy1,2 1Mila and McGill University 2Facebook CIFAR AI Chair {nicholas.meade, elinor.poole-dayan, siva.reddy}@mila.quebec Abstract Recent work has shown pre-trained language models capture social biases from the large amounts of text they are trained on. This has attracted attention to developing techniques that mitigate such biases. In this work, we perform an empirical survey of five recently proposed bias mitigation techniques: Counter- factual Data Augmentation (CDA), Dropout, Iterative Nullspace Projection, Self-Debias, and SentenceDebias. We quantify the effec- tiveness of each technique using three intrinsic bias benchmarks while also measuring the im- pact of these techniques on a model’s language modeling ability, as well as its performance on downstream NLU tasks. We experimentally find that: (1) Self-Debias is the strongest debiasing technique, obtaining improved scores on all bias benchmarks; (2) Current debiasing techniques perform less consistently when mitigating non-gender biases; And (3) improvements on bias benchmarks such as StereoSet and CrowS-Pairs by using debiasing strategies are often accompanied by a decrease in language modeling ability, making it difficult to determine whether the bias mitigation was effective.1 1 Introduction Large pre-trained language models have proven ef- fective across a variety of tasks in natural language processing, often obtaining state of the art perfor- mance (Peters et al., 2018; Devlin et al., 2019; Rad- ford et al., 2019; Brown et al., 2020). These models are typically trained on large amounts of text, orig- inating from unmoderated sources, such as the in- ternet. While the performance of these pre-trained models is remarkable, recent work has shown that they capture social biases from the data they are trained on (May et al. 2019; Kurita et al. 2019; Webster et al. 2020; Nangia et al. 2020; Nadeem 1Our code is publicly available: https://github. com/mcgill-nlp/bias-bench. et al. 2021, inter alia). Because of these findings, an increasing amount of research has focused on de- veloping techniques to mitigate these biases (Liang et al., 2020; Ravfogel et al., 2020; Webster et al., 2020; Kaneko and Bollegala, 2021; Schick et al., 2021; Lauscher et al., 2021). However, the pro- posed techniques are often not investigated thor- oughly. For instance, much work focuses only on mitigating gender bias despite pre-trained language models being plagued by other social biases (e.g., racial or religious bias). Additionally, the impact that debiasing has on both downstream task per- formance, as well as language modeling ability, is often not well explored. In this paper, we perform an empirical survey of the effectiveness of five recently proposed debias- ing techniques for pre-trained language models:2 Counterfactual Data Augmentation (CDA; Zmi- grod et al. 2019; Webster et al. 2020), Dropout (Webster et al., 2020), Iterative Nullspace Projec- tion (INLP; Ravfogel et al. 2020), Self-Debias (Schick et al., 2021), and SentenceDebias (Liang et al., 2020). Following …

Content truncated to 500 words