Is Your Classifier Actually Biased? Measuring Fairness under Uncertainty with Bernstein Bounds

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2914–2919 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2914 Is Your Classifier Actually Biased? Measuring Fairness under Uncertainty with Bernstein Bounds Kawin Ethayarajh Stanford University kawin@stanford.edu Abstract Most NLP datasets are not annotated with pro- tected attributes such as gender, making it dif- ficult to measure classification bias using stan- dard measures of fairness (e.g., equal opportu- nity). However, manually annotating a large dataset with a protected attribute is slow and expensive. Instead of annotating all the exam- ples, can we annotate a subset of them and use that sample to estimate the bias? While it is possible to do so, the smaller this anno- tated sample is, the less certain we are that the estimate is close to the true bias. In this work, we propose using Bernstein bounds to represent this uncertainty about the bias esti- mate as a confidence interval. We provide em- pirical evidence that a 95% confidence inter- val derived this way consistently bounds the true bias. In quantifying this uncertainty, our method, which we call Bernstein-bounded un- fairness, helps prevent classifiers from being deemed biased or unbiased when there is insuf- ficient evidence to make either claim. Our find- ings suggest that the datasets currently used to measure specific biases are too small to conclusively identify bias except in the most egregious cases. For example, consider a co- reference resolution system that is 5% more ac- curate on gender-stereotypical sentences – to claim it is biased with 95% confidence, we need a bias-specific dataset that is 3.8 times larger than WinoBias, the largest available. 1 Introduction NLP models have drawn criticism for capturing common social biases with respect to gender and race (Manzini et al., 2019; Garg et al., 2018; Etha- yarajh, 2019). These biases can be quantified by ap- plying some metric to an embedding space (Boluk- basi et al., 2016), but it is unclear how bias in text embeddings affects decisions made by down- stream classifiers. This is because bias is not prop- agated deterministically: it is possible for mini- mally biased embeddings to be fed into a classi- fier that makes maximally biased predictions (and vice-versa). Moreover, recent work has found that WEAT (Caliskan et al., 2017), the most popular test of embedding bias, can be easily manipulated to claim that bias is present or absent (Ethayarajh et al., 2019a,b). Unlike measuring embedding bias, measuring classification bias is difficult: most NLP datasets are not annotated with protected attributes, preclud- ing the use of standard fairness measures such as equal opportunity (Hardt et al., 2016). However, manually annotating a large dataset with a pro- tected attribute is slow and expensive. In response to this problem, some have created small datasets annotated with a single protected attribute – typi- cally gender – that is used to estimate bias on tasks such as co-reference resolution (Zhao et al., 2018a; Kiritchenko and Mohammad, 2018; Rudinger et al., 2018). This …

Content truncated to 500 words