What’s in a Name? Reducing Bias in Bios without Access to Protected Attributes

Proceedings of NAACL-HLT 2019, pages 4187–4195 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 4187 What’s in a Name? Reducing Bias in Bios without Access to Protected Attributes Alexey Romanov1, Maria De-Arteaga2, Hanna Wallach3, Jennifer Chayes3, Christian Borgs3, Alexandra Chouldechova2, Sahin Geyik4, Krishnaram Kenthapadi4, Anna Rumshisky1, Adam Tauman Kalai3 1University of Massachusetts Lowell {aromanov,arum}@cs.uml.edu 2Carnegie Mellon University mdeartea@andrew.cmu.edu,achould@cmu.edu 3Microsoft Research {wallach,jchayes,Christian.Borgs,Adam.Kalai}@microsoft.com 4LinkedIn {sgeyik,kkenthapadi}@linkedin.com Abstract There is a growing body of work that proposes methods for mitigating bias in machine learn- ing systems. These methods typically rely on access to protected attributes such as race, gen- der, or age. However, this raises two signif- icant challenges: (1) protected attributes may not be available or it may not be legal to use them, and (2) it is often desirable to simulta- neously consider multiple protected attributes, as well as their intersections. In the context of mitigating bias in occupation classification, we propose a method for discouraging corre- lation between the predicted probability of an individual’s true occupation and a word em- bedding of their name. This method leverages the societal biases that are encoded in word embeddings, eliminating the need for access to protected attributes. Crucially, it only re- quires access to individuals’ names at training time and not at deployment time. We evaluate two variations of our proposed method using a large-scale dataset of online biographies. We find that both variations simultaneously reduce race and gender biases, with almost no reduc- tion in the classifier’s overall true positive rate. 1 Introduction In recent years, the performance of machine learning systems has improved substantially, lead- ing to the widespread use of machine learning “What’s in a name? That which we call a rose by any other name would smell as sweet.” – William Shakespeare, Romeo and Juliet. in many domains, including high-stakes domains such as healthcare, employment, and criminal jus- tice (Chalfin et al., 2016; Miotto et al., 2017; Chouldechova, 2017). This increased prevalence has led many people to ask the question, “accurate, but for whom?” (Chouldechova and G’Sell, 2017). When the performance of a machine learning system differs substantially for different groups of people, a number of concerns arise (Baro- cas and Selbst, 2016; Kim, 2016). First and foremost, there is a risk that the deployment of such a method may harm already marginalized groups and widen existing inequalities. Recent work highlights this concern in the context of on- line recruiting and automated hiring (De-Arteaga et al., 2019). When predicting an individual’s occupation from their online biography, the au- thors show that if occupation-specific gender gaps in true positive rates are correlated with exist- ing gender imbalances in those occupations, then those imbalances will be compounded over time— a phenomenon sometimes referred to as the “leaky pipeline.” Second, the correlations that lead to per- formance differences between groups are often ir- relevant. For example, while an occupation clas- sifier should predict a higher probability of soft- ware engineer if an individual’s biography men- tions coding experience, …

Content truncated to 500 words