A Transparent Framework for Evaluating Unintended Demographic Bias in Word Embeddings

A Transparent Framework for Evaluating Unintended Demographic Bias in Word Embeddings Chris Sweeney and Maryam Najafian Massachusetts Institute of Technology Cambridge, MA, USA {csweeney,najafian}@mit.edu Abstract Word embedding models have gained a lot of traction in the Natural Language Processing community, however, they suffer from unin- tended demographic biases. Most approaches to evaluate these biases rely on vector space based metrics like the Word Embedding Asso- ciation Test (WEAT). While these approaches offer great geometric insights into unintended biases in the embedding vector space, they fail to offer an interpretable meaning for how the embeddings could cause discrimination in downstream NLP applications. In this work, we present a transparent framework and met- ric for evaluating discrimination across pro- tected groups with respect to their word em- bedding bias. Our metric (Relative Negative Sentiment Bias, RNSB) measures fairness in word embeddings via the relative negative sen- timent associated with demographic identity terms from various protected groups. We show that our framework and metric enable useful analysis into the bias in word embeddings. 1 Introduction Word embeddings have established themselves as an integral part of Natural Language Processing (NLP) applications. Unfortunately word embed- dings have also introduced unintended biases that could cause downstream NLP systems to be un- fair. Recent studies have shown that word embed- dings exhibit unintended gender and stereotype bi- ases inherent in the training corpus. Bias can be defined as an unfair expression of prejudice for or against a person, a group, or an idea. Bias is a broad term, which covers a range of problems par- ticularly relevant in natural language systems such as, discriminatory gender bias (Bolukbasi et al., 2016a; Zhao et al., 2017), bias against regionally accented speech (Najafian et al., 2016, 2017), per- sonal or political view bias (Iyyer et al., 2014; Re- casens et al., 2013), and many other examples. In Figure 1: 2-D PCA embeddings for positive/negative sentiment words and a set of national origin identity terms. Geometrically, it is difficult to parse how these embeddings can lead to discrimination. our work, we restrict our definition of bias to un- equal distributions of negative sentiment among demographic identity terms in word embeddings. One could also look at unequal distributions of positive sentiment, but for this work we restrict ourselves to the negative case. Sentiment analysis makes up a large portion of current NLP systems. Therefore, preventing neg- ative sentiment from mixing with sensitive at- tributes (i.e. race, gender, religion) in word em- beddings is needed to prevent discrimination in ML models using the embeddings. As studied in (Packer et al., 2018), unintentionally biased word embeddings can have adverse consequences when deployed in applications, such as movie sentiment analyzers or messaging apps. Negative sentiment can be unfairly entangled in the word embeddings, and detecting this unin- tended bias is a difficult problem. We need clear signals to evaluate which groups are discriminated against due to the bias in an embedding model. That way we can pinpoint where to mitigate those biases. To demonstrate this …

Content truncated to 500 words