Robustness and Reliability of Gender Bias Assessment in Word Embeddings: The Role of Base Pairs

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 759–769 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 759 Robustness and Reliability of Gender Bias Assessment in Word Embeddings: The Role of Base Pairs Haiyang Zhang*, Alison Sneyd* and Mark Stevenson Department of Computer Science University of Sheffield haiyang.zhang, a.sneyd, mark.stevenson@sheffield.ac.uk Abstract It has been shown that word embeddings can exhibit gender bias, and various methods have been proposed to quantify this. However, the extent to which the methods are capturing so- cial stereotypes inherited from the data has been debated. Bias is a complex concept and there exist multiple ways to define it. Previ- ous work has leveraged gender word pairs to measure bias and extract biased analogies. We show that the reliance on these gendered pairs has strong limitations: bias measures based off of them are not robust and cannot identify common types of real-world bias, whilst analo- gies utilising them are unsuitable indicators of bias. In particular, the well-known analogy “man is to computer-programmer as woman is to homemaker” is due to word similarity rather than societal bias. This has important impli- cations for work on measuring bias in embed- dings and related work debiasing embeddings. 1 Introduction Word embeddings, distributed representations of words in a low-dimensional vector space, are used in many downstream NLP tasks (Mikolov et al., 2013a,b; Pennington et al., 2014; Peters et al., 2018; Devlin et al., 2019). Recent work has shown they can contain harmful bias and proposed techniques to quantify it (Bolukbasi et al., 2016; Caliskan et al., 2017; Ethayarajh et al., 2019; Gonen and Goldberg, 2019). These techniques leverage co- sine similarity to a base pair of gender words, such as (man, woman). They include bias measures, which return a magnitude of bias for a given word, and analogies. A well-known example of the lat- ter is “Man is to computer programmer as woman is to homemaker” (Bolukbasi et al., 2016), which has been widely interpreted as demonstrating bias. There have also been related attempts to debias * denotes equal contribution. embeddings (Bolukbasi et al., 2016; Zhao et al., 2018; Dev and Phillips, 2019; Kaneko and Bolle- gala, 2019; Manzini et al., 2019). However, to remove bias effectively, an accurate method of identifying it is first required. This is a complex task, not least because the concept of “bias” has multiple interpretations: Mehrabi et al. (2019) identify 23 types of bias that can occur in machine learning applications, including historic (pre-existing in society), algorithmic (introduced by the algorithm) and evaluation (occurs during model evaluation). In the case of word embed- dings, it remains an open question if bias identify- ing techniques reflect social stereotypes in the train- ing data, an artifact of the embedding process or noise. While it is often assumed the first is true, and thus that bias in embeddings can perpetuate harm- ful stereotypes (Bolukbasi et al., 2016; …

Content truncated to 500 words