Understanding Undesirable Word Embedding Associations

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1696–1705 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1696 Understanding Undesirable Word Embedding Associations Kawin Ethayarajh, David Duvenaud†, Graeme Hirst University of Toronto †Vector Institute {kawin, duvenaud, gh}@cs.toronto.edu Abstract Word embeddings are often criticized for cap- turing undesirable word associations such as gender stereotypes. However, methods for measuring and removing such biases remain poorly understood. We show that for any em- bedding model that implicitly does matrix fac- torization, debiasing vectors post hoc using subspace projection (Bolukbasi et al., 2016) is, under certain conditions, equivalent to train- ing on an unbiased corpus. We also prove that WEAT, the most common association test for word embeddings, systematically overesti- mates bias. Given that the subspace projection method is provably effective, we use it to de- rive a new measure of association called the relational inner product association (RIPA). Experiments with RIPA reveal that, on aver- age, skipgram with negative sampling (SGNS) does not make most words any more gendered than they are in the training corpus. However, for gender-stereotyped words, SGNS actually amplifies the gender association in the corpus. 1 Introduction A common criticism of word embeddings is that they capture undesirable associations in vector space. In addition to gender-appropriate analo- gies such as king:queen::man:woman, stereotyp- ical analogies such as doctor:nurse::man:woman also hold in SGNS embedding spaces (Bolukbasi et al., 2016). Caliskan et al. (2017) created an association test for word vectors called WEAT, which uses cosine similarity to measure how as- sociated words are with respect to two sets of at- tribute words (e.g., ‘male’ vs. ‘female’). For ex- ample, they claimed that science-related words were significantly more associated with male at- tributes and art-related words with female ones. Since these associations are socially undesirable, they were described as gender bias. Despite these remarkable findings, such unde- sirable word associations remain poorly under- stood. For one, what causes them – is it biased training data, the embedding model itself, or just noise? Why should WEAT be the test of choice for measuring associations in word embeddings? Bolukbasi et al. (2016) found that word vectors could be debiased by defining a “bias subspace” in the embedding space and then subtracting from each vector its projection on this subspace. But what theoretical guarantee is there that this method actually debiases vectors? In this paper, we answer several of these open questions. We begin by proving that for any em- bedding model that implicitly does matrix fac- torization (e.g., GloVe, SGNS), debiasing vectors post hoc via subspace projection is, under certain conditions, equivalent to training on an unbiased corpus without reconstruction error. We find that contrary to what Bolukbasi et al. (2016) suggested, word embeddings should not be normalized before debiasing, as vector length can contain important information (Ethayarajh et al., 2018). To guaran- tee unbiasedness, the bias subspace should also be the span – rather than a principal component – of the vectors used …

Content truncated to 500 words