Social Biases in NLP Models as Barriers for Persons with Disabilities

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5491–5501 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5491 Social Biases in NLP Models as Barriers for Persons with Disabilities Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, Stephen Denuyl Google {benhutch,vinodkpg,dentone,websterk,yuzhong,sdenuyl}@google.com Abstract Building equitable and inclusive NLP tech- nologies demands consideration of whether and how social attitudes are represented in ML models. In particular, representations en- coded in models often inadvertently perpet- uate undesirable social biases from the data on which they are trained. In this paper, we present evidence of such undesirable biases to- wards mentions of disability in two different English language models: toxicity prediction and sentiment analysis. Next, we demonstrate that the neural embeddings that are the criti- cal first step in most NLP pipelines similarly contain undesirable biases towards mentions of disability. We end by highlighting topical biases in the discourse about disability which may contribute to the observed model biases; for instance, gun violence, homelessness, and drug addiction are over-represented in texts discussing mental illness. 1 Introduction This paper focuses on the representation of persons with disabilities through the lens of technology. Specifically, we examine how NLP models classify or predict text relating to persons with disabilities (see Table 1). This is important because NLP mod- els are increasingly being used for tasks such as fighting online abuse (Jigsaw, 2017), measuring brand sentiment (Mostafa, 2013), and matching job applicants to job opportunities (De-Arteaga et al., 2019). In addition, since text classifiers are trained on large datasets, the biases they exhibit may be indicative of societal perceptions of persons with disabilities (Caliskan et al., 2017). If models in- appropriately condition on mentions of disability, this could impact people writing, reading, or seek- ing information about a disability. Depending on how such models are deployed, this could poten- tially result in reduced autonomy, reduced freedom Sentence Toxicity I am a person with mental illness. 0.62 I am a deaf person. 0.44 I am a blind person. 0.39 I am a tall person. 0.03 I am a person. 0.08 I will fight for people with mental illnesses. 0.54 I will fight for people who are deaf. 0.42 I will fight for people who are blind. 0.29 I will fight for people. 0.14 Table 1: Example toxicity scores from Perspective API. of speech, perpetuation of societal stereotypes or inequities, or harms to the dignity of individuals. While previous studies have studied unintended biases in NLP models against other historically marginalized groups (Bolukbasi et al., 2016; Caliskan et al., 2017; Garg et al., 2017; Barocas et al., 2017; Garg et al., 2019; Dixon et al., 2018; Noble, 2018; Manzini et al., 2019; Sap et al., 2019; May et al., 2019; Speer, 2017), bias with respect to different disability groups has been relatively under-explored. However, over one billion indi- viduals (about 15% of the world’s population) are persons with disabilities,1 and disability is some- times the subject of strong negative social biases. …

Content truncated to 500 words