Measuring Fairness of Text Classifiers via Prediction Sensitivity

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5830 - 5842 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Measuring Fairness of Text Classifiers via Prediction Sensitivity Satyapriya Krishna ∗ Harvard University skrishna@g.harvard.edu Jwala Dhamala Amazon Alexa AI jddhamal@amazon.com Rahul Gupta Amazon Alexa AI gupra@amazon.com Yada Pruksachatkun ∗ Amazon Alexa AI yada.pruksachatkun@gmail.com Apurv Verma Amazon Alexa AI verapurv@amazon.com Kai-Wei Chang UCLA, Amazon Alexa kwchang@cs.ucla.edu Abstract With the rapid growth in language processing applications, fairness has emerged as an impor- tant consideration in data-driven solutions. Al- though various fairness definitions have been explored in the recent literature, there is lack of consensus on which metrics most accu- rately reflect the fairness of a system. In this work, we propose a new formulation – ACCU- MULATED PREDICTION SENSITIVITY, which measures fairness in machine learning models based on the model’s prediction sensitivity to perturbations in input features. The metric at- tempts to quantify the extent to which a sin- gle prediction depends on a protected attribute, where the protected attribute encodes the mem- bership status of an individual in a protected group. We show that the metric can be theo- retically linked with a specific notion of group fairness (statistical parity) and individual fair- ness. It also correlates well with humans’ per- ception of fairness. We conduct experiments on two text classification datasets – JIGSAW TOXICITY, and BIAS IN BIOS, and evaluate the correlations between metrics and manual annotations on whether the model produced a fair outcome. We observe that the proposed fairness metric based on prediction sensitiv- ity is statistically significantly more correlated with human annotation than the existing coun- terfactual fairness metric. 1 Introduction Ongoing research is increasingly emphasizing the development of methods which detect and miti- gate unfair social bias present in machine learning- based language processing models. These methods come under the umbrella of algorithmic fairness which has been quantitatively expressed with nu- merous definitions (Mehrabi et al., 2019; Jacobs and Wallach, 2019). These fairness definitions are ∗* Work done while working at Amazon broadly categorized into two types, i.e, individual fairness and group fairness. Individual fairness (e.g., counter-factual fairness (Kusner et al., 2017)) is aimed at evaluating whether a model gives simi- lar predictions for individuals with similar personal attributes (e.g., age or race). On the other hand, group fairness (e.g., statistical parity (Dwork et al., 2012)) evaluates fairness across cohorts with same protected attributes instead of individuals (Mehrabi et al., 2019). Although these two broad categories of fairness define valid notions of fairness, hu- man understanding of fairness is also used to mea- sure fairness in machine learning models (Dhamala et al., 2021). Existing studies often consider only one or two these verticals of measuring fairness. In our work, we propose a formulation based on models sensitivity to input features – the accumu- lated prediction sensitivity, to measure fairness of model predictions. We establish its theoretical rela- tionship with statistical parity (group fairness) and individual fairness (Dwork et …

Content truncated to 500 words