Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics

Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics Paula Czarnowska♠ University of Cambridge, UK pjc211@cam.ac.uk Yogarshi Vyas Amazon AI, USA yogarshi@amazon.com Kashif Shah Amazon AI, USA shahkas@amazon.com Abstract Measuring bias is key for better understanding and addressing unfairness in NLP/ML models. This is often done via fairness metrics, which quantify the differences in a model’s behaviour across a range of demographic groups. In this work, we shed more light on the differences and similarities between the fairness metrics used in NLP. First, we unify a broad range of existing metrics under three generalized fairness metrics, revealing the connections be- tween them. Next, we carry out an extensive empirical comparison of existing metrics and demonstrate that the observed differences in bias measurement can be systematically ex- plained via differences in parameter choices for our generalized metrics. 1 Introduction The prevalence of unintended social biases in NLP models has been recently identified as a major concern for the field. A number of papers have published evidence of uneven treatment of different demographics (Dixon et al., 2018; Zhao et al., 2018; Rudinger et al., 2018; Garg et al., 2019; Borkan et al., 2019; Stanovsky et al., 2019; Gonen and Webster, 2020; Huang et al., 2020a; Nangia et al., 2020), which can reportedly cause a variety of serious harms, like unfair allocation of opportunities or unfavorable representation of particular social groups (Blodgett et al., 2020). Measuring bias in NLP models is key for better understanding and addressing unfairness. This is often done via fairness metrics, which quantify the differences in a model’s behavior across a range of social groups. The community has pro- posed a multitude of such metrics (Dixon et al., 2018; Garg et al., 2019; Huang et al., 2020a; Borkan et al., 2019; Gaut et al., 2020). In this pa- ♠ Work done during an internship at Amazon AI. per, we aim to shed more light on how those varied means of quantifying bias differ and what facets of bias they capture. Developing such understand- ing is crucial for drawing reliable conclusions and actionable recommendations regarding bias. We focus on bias measurement for downstream tasks, as Goldfarb-Tarrant et al. (2021) have recently shown that there is no reliable correlation between bias measured intrinsically on, for example, word embeddings, and bias measured extrinsically on a downstream task. We narrow down the scope of this paper to tasks that do not involve prediction of a sensitive attribute. We survey 146 papers on social bias in NLP and unify the multitude of disparate metrics we find under three generalized fairness metrics. Through this unification we reveal the key connec- tions between a wide range of existing metrics— we show that they are simply different param- etrizations of our generalized metrics. Next, we empirically investigate the role of different metrics in detecting the systemic differences in perfor- mance for different demographic groups, namely, differences in quality of service (Jacobs et al., 2020). We experiment on three transformer-based models—two models for …

Content truncated to 500 words