Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems

Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (*SEM), pages 43–53 New Orleans, June 5-6, 2018. c⃝2018 Association for Computational Linguistics Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems Svetlana Kiritchenko and Saif M. Mohammad National Research Council Canada {svetlana.kiritchenko,saif.mohammad}@nrc-cnrc.gc.ca Abstract Automatic machine learning systems can in- advertently accentuate and perpetuate inappro- priate human biases. Past work on examin- ing inappropriate biases has largely focused on just individual systems. Further, there is no benchmark dataset for examining inappro- priate biases in systems. Here for the first time, we present the Equity Evaluation Cor- pus (EEC), which consists of 8,640 English sentences carefully chosen to tease out biases towards certain races and genders. We use the dataset to examine 219 automatic senti- ment analysis systems that took part in a re- cent shared task, SemEval-2018 Task 1 ‘Affect in Tweets’. We find that several of the systems show statistically significant bias; that is, they consistently provide slightly higher sentiment intensity predictions for one race or one gen- der. We make the EEC freely available. 1 Introduction Automatic systems have had a significant and ben- eficial impact on all walks of human life. So much so that it is easy to overlook their potential to benefit society by promoting equity, diversity, and fairness. For example, machines do not take bribes to do their jobs, they can determine eligi- bility for a loan without being influenced by the color of the applicant’s skin, and they can pro- vide access to information and services without discrimination based on gender or sexual orien- tation. Nonetheless, as machine learning systems become more human-like in their predictions, they can also perpetuate human biases. Some learned biases may be beneficial for the downstream appli- cation (e.g., learning that humans often use some insect names, such as spider or cockroach, to refer to unpleasant situations). Other biases can be in- appropriate and result in negative experiences for some groups of people. Examples include, loan el- igibility and crime recidivism prediction systems that negatively assess people belonging to a cer- tain pin/zip code (which may disproportionately impact people of a certain race) (Chouldechova, 2017) and resum´e sorting systems that believe that men are more qualified to be programmers than women (Bolukbasi et al., 2016). Similarly, senti- ment and emotion analysis systems can also per- petuate and accentuate inappropriate human bi- ases, e.g., systems that consider utterances from one race or gender to be less positive simply be- cause of their race or gender, or customer support systems that prioritize a call from an angry male over a call from the equally angry female. Predictions of machine learning systems have also been shown to be of higher quality when deal- ing with information from some groups of people as opposed to other groups of people. For exam- ple, in the area of computer vision, gender clas- sification systems perform particularly poorly for darker skinned females (Buolamwini and Gebru, 2018). Natural language processing (NLP) sys- tems …

Content truncated to 500 words