On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3770–3783 June 6–11, 2021. ©2021 Association for Computational Linguistics 3770 On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning Xisen Jin§, Francesco Barbieri†, Brendan Kennedy§, Aida Mostafazadeh Davani§, Leonardo Neves†, Xiang Ren§ §University of Southern California †Snap Inc. {xisenjin, mostafaz, btkenned, xiangren}@usc.edu {fbarbieri, lneves}@snap.com Abstract Fine-tuned language models have been shown to exhibit biases against protected groups in a host of modeling tasks such as text classi- fication and coreference resolution. Previous works focus on detecting these biases, reduc- ing bias in data representations, and using aux- iliary training objectives to mitigate bias dur- ing fine-tuning. Although these techniques achieve bias reduction for the task and domain at hand, the effects of bias mitigation may not directly transfer to new tasks, requiring addi- tional data collection and customized annota- tion of sensitive attributes, and re-evaluation of appropriate fairness metrics. We explore the feasibility and benefits of upstream bias mitigation (UBM) for reducing bias on down- stream tasks, by first applying bias mitiga- tion to an upstream model through fine-tuning and subsequently using it for downstream fine- tuning. We find, in extensive experiments across hate speech detection, toxicity detec- tion, occupation prediction, and coreference resolution tasks over various bias factors, that the effects of UBM are indeed transferable to new downstream tasks or domains via fine- tuning, creating less biased downstream mod- els than directly fine-tuning on the down- stream task or transferring from a vanilla up- stream model. Though challenges remain, we show that UBM promises more efficient and accessible bias mitigation in LM fine-tuning.12 1 Introduction The practice of fine-tuning pretrained language models (PTLMs or LMs), such as BERT (Devlin et al., 2019), has improved prediction performance in a wide range of NLP tasks. However, fine- tuned LMs may exhibit biases against certain pro- tected groups (e.g., gender and ethnic minorities), 1Code and data: https://github.com/INK-U SC/Upstream-Bias-Mitigation 2The work was partially done when Xisen Jin was an intern at Snap Inc. (a) Classifier bias mitigation (b) Representational bias mitigation Model (c) Removing sensitive attributes from data De-bias Dataset Dataset .man .woman .doctor De-bias Model Model … De-bias Upstream Model Model Dataset 𝒟!" Dataset 𝒟!# (d) Upstream bias mitigation for fine-tuning Dataset Data representations … Frozen Model Fine-tuning Datasets 𝒟$", 𝒟$# … De-bias Model Figure 1: Comparison between the focus of our study (d) and previous works (a,b,c). We study the viability of obtaining an upstream model that could re- duce bias in a number of downstream classifiers when fine-tuned. as models may learn to associate certain features with positive or negative labels spuriously (Dixon et al., 2018), or propagate bias encoded in PTLMs to downstream classifiers (Caliskan et al., 2017; Bolukbasi et al., 2016). Among many exam- ples, Kurita et al. (2019) demonstrates gender-bias in the pronoun resolution task when models are trained using BERT embeddings, and Kennedy et al. (2020) shows that hate speech classifiers fine- tuned from …

Content truncated to 500 words