Marked Attribute Bias in Natural Language Inference

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4214–4226 August 1–6, 2021. ©2021 Association for Computational Linguistics 4214 Marked Attribute Bias in Natural Language Inference Hillary Dawkins University of Guelph, Ontario, Canada Vector Institute, Toronto, Ontario, Canada hdawkins@uoguelph.ca Abstract Reporting and providing test sets for harm- ful bias in NLP applications is essential for building a robust understanding of the cur- rent problem. We present a new observation of gender bias in a downstream NLP appli- cation: marked attribute bias in natural lan- guage inference. Bias in downstream appli- cations can stem from training data, word embeddings, or be amplified by the model in use. However, focusing on biased word embeddings is potentially the most impact- ful first step due to their universal nature. Here we seek to understand how the intrin- sic properties of word embeddings contribute to this observed marked attribute effect, and whether current post-processing methods ad- dress the bias successfully. An investiga- tion of the current debiasing landscape reveals two open problems: none of the current de- biased embeddings mitigate the marked at- tribute error, and none of the intrinsic bias mea- sures are predictive of the marked attribute ef- fect. By noticing that a new type of intrinsic bias measure correlates meaningfully with the marked attribute effect, we propose a new post- processing debiasing scheme for static word embeddings. The proposed method applied to existing embeddings achieves new best re- sults on the marked attribute bias test set. See https://github.com/hillary-dawkins/MAB. 1 Introduction Pre-trained distributed representations of words (a.k.a. word embeddings) are ubiquitous tools in natural language processing (NLP). Their utility is owing to the remarkable success in mapping se- mantic and syntactic relationships among words to linear relationships among real-valued vectors. For instance, analogy generation using vector addition on word embeddings (e.g. Tokyo is to Japan as Paris is to France) was taken to be an early mea- sure of word embedding quality. In all kinds of related tasks, the vector space is known to encode semantic meaning surprisingly well (Pennington et al., 2014; Mikolov et al., 2013b,c). However, harmful gender-biased properties of word embed- dings are also known to exist. Later is was observed that the same analogy generation property that pro- duced the celebrated “man is to king as woman is to queen” analogy would also predict “man is to programmer as woman is to homemaker” (Boluk- basi et al., 2016). This observation sparked interest in developing debiased word embeddings. Post-processing debiasing schemes are usually motivated by recognizing some intrinsic measure of bias in the embedding space, and then attempting to reduce that intrinsic bias. Early work (2016- 2017) focused on the idea of a “gender direction” vector within the embedding space, loosely defined as the difference vector between female and male attribute words. It was noted that any non-zero pro- jection of a word onto the gender direction (termed direct bias) implied that the word was more related to one gender over another. In the case of ideally gender-neutral words …

Content truncated to 500 words