Detecting Independent Pronoun Bias with Partially-Synthetic Data Generation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2011–2017, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2011 Detecting Independent Pronoun Bias with Partially-Synthetic Data Generation Robert (Munro) Monarch Machine Learning Consulting San Francisco, CA rmunro@alumni.stanford.edu Alex (Carmen) Morrison Amazon Web Services Oakland, CA Abstract We report that state-of-the-art parsers consis- tently failed to identify “hers” and “theirs” as pronouns but identified the masculine equiva- lent “his”. We find that the same biases exist in recent language models like BERT. While some of the bias comes from known sources, like training data with gender imbalances, we find that the bias is amplified in the language models and that linguistic differences between English pronouns that are not inherently bi- ased can become biases in some machine learn- ing models. We introduce a new technique for measuring bias in models, using Bayesian approximations to generate partially-synthetic data from the model itself. 1 Introduction We share a negative result for the Natural Language Processing (NLP) community as a whole: for 20 years the major part-of-speech (POS) taggers and parsers missed that “hers” and “theirs” were pro- nouns, but it had gone unreported until this paper. This paper also shows that biases against “hers” and “theirs” are amplified in popular language models, predicted by the models with less frequency than expected given the training data. Our solution for the parser problem is a new dataset with “hers” and “theirs” used in a syntacti- cally diverse set of contexts, released in Universal Dependency format (Nivre et al., 2016). For the language model problem, we introduce a novel use of Bayesian modeling for sentence gener- ation, in our case using it to detect bias by alternat- ing pronouns in different contexts. The contexts are suggested by the model, avoiding the problems in measuring bias that come from rare or pathological data (Feng et al., 2018). We conclude that this is a general technique that can be used for measuring other types of bias and for text generation more broadly. While our contribution doesn’t mitigate bias in language models, it improves the ability to detect and measure bias. We test on BERT (Devlin et al., 2019) because it is the most widely used pretrained model for which all the training data was also avail- able. We find that other masked language models are also amplifying the bias in their data, but we cannot measure how much that bias is amplified without access to the training data. Where data for a pretrained model can’t be shared, we encourage researchers to report on the biases in their models using the techniques in this paper. 1.1 The bias is hers...and singular theirs Independent possessive pronouns (IDPs) are inter- esting problems for NLP because they are the only English pronouns to encode two long-distance re- lationships: the person possessing an attribute and the attribute being possessed (see Table 1). The preference for “his” in language models will bias any text generation system against “hers” or “theirs”, …

Content truncated to 500 words