Mitigating Gender Bias Amplification in Distribution by Posterior Regularization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2936–2942 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2936 Mitigating Gender Bias Amplification in Distribution by Posterior Regularization Shengyu Jia♣∗, Tao Meng♠∗, Jieyu Zhao♠, Kai-Wei Chang♠ ♣ Tsinghua University ♠ University of California, Los Angeles jiasy16@mails.tsinghua.edu.cn, {mengt18, jieyuzhao, kwchang}@ucla.edu Abstract Advanced machine learning techniques have boosted the performance of natural language processing. Nevertheless, recent studies, e.g., Zhao et al. (2017) show that these techniques inadvertently capture the societal bias hidden in the corpus and further amplify it. However, their analysis is conducted only on models’ top predictions. In this paper, we investigate the gender bias amplification issue from the dis- tribution perspective and demonstrate that the bias is amplified in the view of predicted prob- ability distribution over labels. We further pro- pose a bias mitigation approach based on pos- terior regularization. With little performance loss, our method can almost remove the bias amplification in the distribution. Our study sheds the light on understanding the bias am- plification. 1 Introduction Data-driven machine learning models have achieved high performance in various applications. Despite the impressive results, recent studies (e.g., Wang et al. (2019); Hendricks et al. (2018)) demon- strate that these models may carry societal biases exhibited in the dataset they trained on. In particu- lar, Zhao et al. (2017) show that a model trained on a biased dataset may amplify the bias. For exam- ple, we can consider a task of labeling the activity and objects depicted in an image. The training set contains 30% more images with “woman cooking” than “man cooking”. However, when evaluating the top predictions of a trained model, the disparity between males and females is amplified to around 70%. Based on this observation, Zhao et al. (2017) conduct a systematic study and propose to calibrate the top predictions of a learned model by injecting ∗Both authors contributed equally to this work and are listed in alphabetical order. corpus-level constraints to ensure that the gender disparity is not amplified. However, when analyzing the top predictions, the models are forced to make one decision. There- fore, even if the model assigns high scores to both labels of “woman cooking” and “man cooking”, it has to pick one as the prediction. This process obviously has a risk to amplify the bias. However, to our surprise, we observe that gender bias is also amplified when analyzing the posterior distribution of the predictions. Since the model is trained with regularized maximal likelihood objective, the bias in distribution is a more fundamental perspective of analyzing the bias amplification issue. In this paper, we conduct a systematic study to quantify the bias in the predicted distribution over labels. Our analysis demonstrates that when evalu- ating the distribution, though not as significant as when evaluating top predictions, the bias amplifica- tion exists. About half of activities show significant bias amplification in the posterior distribution, and on average, they amplify the bias by 3.2%. We further propose a …

Content truncated to 500 words