Debiasing Isn’t Enough! – on the Effectiveness of Debiasing MLMs and Their Social Biases in Downstream Tasks

Proceedings of the 29th International Conference on Computational Linguistics, pages 1299–1310 October 12–17, 2022. 1299 Debiasing isn’t enough! – On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks Masahiro Kaneko1 Danushka Bollegala2,3∗ Naoaki Okazaki1 1Tokyo Institute of Technology 2University of Liverpool masahiro.kaneko@nlp.c.titech.ac.jp3Amazon danushka@liverpool.ac.uk okazaki@c.titech.ac.jp Abstract We study the relationship between task- agnostic intrinsic and task-specific extrinsic social bias evaluation measures for Masked Language Models (MLMs), and find that there exists only a weak correlation between these two types of evaluation measures. Moreover, we find that MLMs debiased using different methods still re-learn social biases during fine- tuning on downstream tasks. We identify the social biases in both training instances as well as their assigned labels as reasons for the dis- crepancy between intrinsic and extrinsic bias evaluation measurements. Overall, our findings highlight the limitations of existing MLM bias evaluation measures and raise concerns on the deployment of MLMs in downstream applica- tions using those measures. 1 Introduction Text representations produced by MLMs have rev- olutionised NLP by improving the performance of numerous downstream applications (Peters et al., 2018; Devlin et al., 2019; Liu et al., 2019; Lan et al., 2020; Yang et al., 2019). Unfortunately, large-scale pretrained MLMs demonstrate worry- ing levels of social biases when applied to down- stream tasks (May et al., 2019; Nadeem et al., 2021; Kaneko and Bollegala, 2021a; Kaneko et al., 2022b). Given that real-world Natural Language Processing (NLP) systems such as machine trans- lation systems, dialogue systems, etc. are used by millions of users world-wide (Hovy and Spruit, 2016), it remains an important responsibility to ac- curately evaluate the social biases in MLMs prior to deployment. Two types of social bias evaluation mea- sures have been proposed for MLMs in prior work (Goldfarb-Tarrant et al., 2021; Cao et al., ∗Danushka Bollegala holds concurrent appointments as a Professor at University of Liverpool and as an Amazon Scholar. This paper describes work performed at the Univer- sity of Liverpool and is not associated with Amazon. 2022): (a) task-agnostic intrinsic evaluation mea- sures (Nangia et al., 2020; Nadeem et al., 2021; Kaneko and Bollegala, 2022) that use the likeli- hood scores assigned by an MLM under evaluation for sentences representing various social biases, and (b) task-specific extrinsic evaluation measures that use the data from downstream NLP tasks such as predicting the occupation of a person from their biographies (De-Arteaga et al., 2019; Bartl et al., 2020). In contrast to intrinsic measures that directly probe into the biases in MLMs, extrinsic measures holistically evaluate an NLP system that uses an MLM. Much prior work has decoupled the intrinsic and extrinsic bias evaluations for simplicity, and have assumed that an MLM unbiased according to an intrinsic measure will remain to be so under ex- trinsic evaluations too. However, this unverified hypothesis begs the following question: Can we reliably predict the social biases of an MLM when it is applied to a particular downstream task using only intrinsic evaluation measures? To answer this question, we conduct …

Content truncated to 500 words