Measuring Fairness with Biased Rulers: A Comparative Study on Bias Metrics for Pre-trained Language Models

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1693 - 1706 July 10-15, 2022 ©2022 Association for Computational Linguistics Measuring Fairness with Biased Rulers: A Comparative Study on Bias Metrics for Pre-trained Language Models Pieter Delobelle1, Ewoenam Kwaku Tokpo2, Toon Calders2 and Bettina Berendt1,3 1 Department of Computer Science, KU Leuven; Leuven.AI 2 Department of Computer Science, University of Antwerp 3 Faculty of Electrical Engineering and Computer Science, TU Berlin; Weizenbaum Institute Abstract An increasing awareness of biased patterns in natural language processing resources such as BERT has motivated many metrics to quantify ‘bias’ and ‘fairness’ in these resources. How- ever, comparing the results of different metrics and the works that evaluate with such metrics remains difficult, if not outright impossible. We survey the literature on fairness metrics for pre-trained language models and experimen- tally evaluate compatibility, including both bi- ases in language models and in their down- stream tasks. We do this by combining tra- ditional literature survey, correlation analysis and empirical evaluations. We find that many metrics are not compatible with each other and highly depend on (i) templates, (ii) attribute and target seeds and (iii) the choice of embed- dings. We also see no tangible evidence of in- trinsic bias relating to extrinsic bias. These re- sults indicate that fairness or bias evaluation re- mains challenging for contextualized language models, among other reasons because these choices remain subjective. To improve fu- ture comparisons and fairness evaluations, we recommend to avoid embedding-based metrics and focus on fairness evaluations in down- stream tasks. 1 Introduction With the popularization of word embeddings by works such as Word2vec (Mikolov et al., 2013), GLoVe (Pennington et al., 2014) and, more re- cently, contextualized variants such as ELMo (Pe- ters et al., 2018) and BERT (Devlin et al., 2019), Natural Language Processing (NLP) has seen sig- nificant growth and advancement. Word embed- dings and later language models have been adopted by many applications. Many of these embeddings have been probed by researchers for biases such as gender stereotypes. Word embeddings are generally trained on real- world data such that they model statistical proper- ties from the training data. Hence, they pick up biases and stereotypes that are typically present in the data (Garrido-Mu˜noz et al., 2021). Al- though Kurita et al. (2019) and Webster et al. (2020) opine that this can pose significant challenges in downstream applications, this view has been ques- tioned, especially for non-contextualized word em- beddings (Goldfarb-Tarrant et al., 2021). Early works such as Bolukbasi et al. (2016); Caliskan et al. (2017); Gonen and Goldberg (2019) widely explored fairness in non-contextualized em- bedding methods. In non-contextualized embed- dings such as Word2vec and GLoVe embeddings, models are trained to generate vectors that map directly to dictionary words and hence are inde- pendent of the context in which the word is used. In contrast, contextualized word embeddings take polysemy (words could have multiple meanings, e.g. ‘a stick’ vs ‘let’s stick …

Content truncated to 500 words