Gender Bias in Contextualized Word Embeddings

Proceedings of NAACL-HLT 2019, pages 629–634 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 629 Gender Bias in Contextualized Word Embeddings Jieyu Zhao§ Tianlu Wang† Mark Yatskar‡ Ryan Cotterellℵ Vicente Ordonez† Kai-Wei Chang§ §University of California, Los Angeles {jyzhao, kwchang}@cs.ucla.edu †University of Virginia {tw8bc, vicente}@virginia.edu ‡Allen Institute for Artificial Intelligence marky@allenai.org ℵUniversity of Cambridge rdc42@cam.ac.uk Abstract In this paper, we quantify, analyze and miti- gate gender bias exhibited in ELMo’s contex- tualized word vectors. First, we conduct sev- eral intrinsic analyses and find that (1) train- ing data for ELMo contains significantly more male than female entities, (2) the trained ELMo embeddings systematically encode gender in- formation and (3) ELMo unequally encodes gender information about male and female en- tities. Then, we show that a state-of-the-art coreference system that depends on ELMo in- herits its bias and demonstrates significant bias on the WinoBias probing corpus. Finally, we explore two methods to mitigate such gender bias and show that the bias demonstrated on WinoBias can be eliminated. 1 Introduction Distributed representations of words in the form of word embeddings (Mikolov et al., 2013; Pen- nington et al., 2014) and contextualized word em- beddings (Peters et al., 2018; Devlin et al., 2018; Radford et al., 2018; McCann et al., 2017; Radford et al., 2019) have led to huge performance improve- ment on many NLP tasks. However, several re- cent studies show that training word embeddings in large corpora could lead to encoding societal biases present in these human-produced data (Bolukbasi et al., 2016; Caliskan et al., 2017). In this work, we extend these analyses to the ELMo contextualized word embeddings. Our work provides a new intrinsic analysis of how ELMo represents gender in biased ways. First, the corpus used for training ELMo has a significant gender skew: male entities are nearly three times more common than female entities, which leads to gender bias in the downloadable pre-trained con- textualized embeddings. Then, we apply princi- pal component analysis (PCA) to show that after training on such biased corpora, there exists a low- dimensional subspace that captures much of the gender information in the contextualized embed- dings. Finally, we evaluate how faithfully ELMo preserves gender information in sentences by mea- suring how predictable gender is from ELMo repre- sentations of occupation words that co-occur with gender revealing pronouns. Our results show that ELMo embeddings perform unequally on male and female pronouns: male entities can be predicted from occupation words 14% more accurately than female entities. In addition, we examine how gender bias in ELMo propagates to the downstream applications. Specifically, we evaluate a state-of-the-art coref- erence resolution system (Lee et al., 2018) that makes use of ELMo’s contextual embeddings on WinoBias (Zhao et al., 2018a), a coreference di- agnostic dataset that evaluates whether systems behave differently on decisions involving male and female entities of stereotyped or anti-stereotyped occupations. We find that in the most challenging setting, the ELMo-based system has a disparity in accuracy between pro- and anti-stereotypical pre- …

Content truncated to 500 words