BLIND: Bias Removal With No Demographics

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8801–8821 July 9-14, 2023 ©2023 Association for Computational Linguistics BLIND: Bias Removal With No Demographics Hadas Orgad Yonatan Belinkov∗ orgad.hadas@cs.technion.ac.il belinkov@technion.ac.il Technion – Israel Institute of Technology Abstract Models trained on real-world data tend to imi- tate and amplify social biases. Common meth- ods to mitigate biases require prior informa- tion on the types of biases that should be miti- gated (e.g., gender or racial bias) and the social groups associated with each data sample. In this work, we introduce BLIND, a method for bias removal with no prior knowledge of the demographics in the dataset. While training a model on a downstream task, BLIND detects biased samples using an auxiliary model that predicts the main model’s success, and down- weights those samples during the training pro- cess. Experiments with racial and gender biases in sentiment classification and occupation clas- sification tasks demonstrate that BLIND miti- gates social biases without relying on a costly demographic annotation process. Our method is competitive with other methods that require demographic information and sometimes even surpasses them.1 1 Introduction Neural natural language processing (NLP) mod- els are known to suffer from social biases, such as performance disparities between genders or races (Blodgett et al., 2020). Numerous debiasing meth- ods have been proposed in order to address this issue, with varying degrees of success. A disadvan- tage of these methods is that they require knowl- edge of the biases one wishes to mitigate (e.g., gender bias) (Bolukbasi et al., 2016; Zhao et al., 2018; De-Arteaga et al., 2019; Maudslay et al., 2019). Moreover, some methods require additional annotations for identifying the demographics for each sample in the data, such as the race of the writer (Han et al., 2021a; Liu et al., 2021; Ravfogel et al., 2022; Shen et al., 2022). Some annotations ∗Supported by the Viterbi Fellowship in the Center for Computer Engineering at the Technion. 1Our code is available at https://github.com/ technion-cs-nlp/BLIND. Feature Extractor Classification Layer Gender Detector End-to-end finetuning A female nurse biography 0.9 0.1 Female Male 0.05 0.7 0.2 … DFL loss (a) With demographic annotations. Demographics detector learns to predict the demographic data, e.g., gender. Feature Extractor Classification Layer Success Detector End-to-end finetuning A nurse biography, gender is unknown 0.7 0.3 Wrong Correct 0.05 0.7 0.2 … DFL loss (b) BLIND: Without demographic annotations. Success detec- tor learns to predict when the main model is correct. Supervi- sion is based only on the downstream task labels. Forward pass Backward pass Figure 1: Our proposed debiasing methods. In both cases, an auxiliary classifier is trained to detect sam- ples where demographic features may be used as short- cuts and their importance to the main model is down- weighted. can be automatically collected from the data, while others require manual annotations or expert knowl- edge, which can be very costly. Thus, existing methods are typically limited to a small number of datasets and tasks. In …

Content truncated to 500 words