On Measuring Social Biases in Sentence Encoders

Proceedings of NAACL-HLT 2019, pages 622–628 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 622 On Measuring Social Biases in Sentence Encoders Chandler May1 Alex Wang2 Shikha Bordia2 Samuel R. Bowman2 Rachel Rudinger1 1Johns Hopkins University 2New York University {cjmay,rudinger}@jhu.edu {alexwang,sb6416,bowman}@nyu.edu Abstract The Word Embedding Association Test shows that GloVe and word2vec word embed- dings exhibit human-like implicit biases based on gender, race, and other social con- structs (Caliskan et al., 2017). Meanwhile, research on learning reusable text representa- tions has begun to explore sentence-level texts, with some sentence encoders seeing enthusi- astic adoption. Accordingly, we extend the Word Embedding Association Test to measure bias in sentence encoders. We then test sev- eral sentence encoders, including state-of-the- art methods such as ELMo and BERT, for the social biases studied in prior work and two im- portant biases that are difficult or impossible to test at the word level. We observe mixed re- sults including suspicious patterns of sensitiv- ity that suggest the test’s assumptions may not hold in general. We conclude by proposing di- rections for future work on measuring bias in sentence encoders. 1 Introduction Word embeddings quickly achieved wide adoption in natural language processing (NLP), precipitat- ing the development of efficient, word-level neural models of human language. However, prominent word embeddings such as word2vec (Mikolov et al., 2013) and GloVe (Pennington et al., 2014) encode systematic biases against women and black people (Bolukbasi et al., 2016; Garg et al., 2018, i.a.), implicating many NLP systems in scal- ing up social injustice. We investigate whether sentence encoders, which extend the word embed- ding approach to sentences, are similarly biased.1 The previously developed Word Embedding Association Test (WEAT; Caliskan et al., 2017) measures bias in word embeddings by comparing two sets of target-concept words to two sets of at- tribute words. We propose a simple generaliza- 1 While encoder training data may contain perspectives from outside the U.S., we focus on biases in U.S. contexts. tion of WEAT to phrases and sentences: the Sen- tence Encoder Association Test (SEAT). We apply SEAT to sentences generated by inserting individ- ual words from Caliskan et al.’s tests into simple templates such as “This is a[n] <word>.” To demonstrate the new potential of a sentence- level approach and advance the discourse on bias in NLP, we also introduce tests of two biases that are less amenable to word-level representation: the angry black woman stereotype (Collins, 2004; Madison, 2009; Harris-Perry, 2011; hooks, 2015; Gillespie, 2016) and a double bind on women in professional settings (Heilman et al., 2004). The use of sentence-level contexts also facili- tates testing the impact of different experimental designs. For example, several of Caliskan et al.’s tests rely on given names associated with Euro- pean American and African American people or rely on terms referring to women and men as groups (such as “woman” and “man”). We explore the effect of using given names versus group terms by creating alternate versions of several bias tests …

Content truncated to 500 words