Adversarial Removal of Demographic Attributes Revisited

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6330–6335, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6330 Adversarial Removal of Demographic Attributes Revisited Maria Barrett1, Yova Kementchedjhieva1, Yanai Elazar2, Desmond Elliott1, Anders Søgaard1 1University of Copenhagen, 2Bar-Ilan University {mjb,yova,de,soegaard}@di.ku.dk,yanaiela@gmail.com Abstract Elazar and Goldberg (2018) showed that pro- tected attributes can be extracted from the rep- resentations of a debiased neural network for mention detection at above-chance levels, by evaluating a diagnostic classifier on a held- out subsample of the data it was trained on. We revisit their experiments and conduct a se- ries of follow-up experiments showing that, in fact, the diagnostic classifier generalizes poorly to both new in-domain samples and new domains, indicating that it relies on corre- lations specific to their particular data sample. We further show that a diagnostic classifier trained on the biased baseline neural network also does not generalize to new samples. In other words, the biases detected in Elazar and Goldberg (2018) seem restricted to their par- ticular data sample, and would therefore not bias the decisions of the model on new sam- ples, whether in-domain or out-of-domain. In light of this, we discuss better methodologies for detecting bias in our models. 1 Introduction Several approaches have been proposed to learn classifiers that are invariant (unbiased with re- spect) to protected attributes: cost-sensitive (Agar- wal et al., 2018), regularization-based (Bechavod and Ligett, 2017), and adversarial (Ganin and Lempitsky, 2015). In the adversarial approach, a model learns representations x that should be pre- dictive for a main task y and oblivious to a pro- tected attribute z. Adversarial training has been used to learn data representations that are invariant to demographic attributes (Raff and Sylvester, 2018; Beutel et al., 2017; Li et al., 2018), as well as representa- tions invariant to domain differences (Ganin and Lempitsky, 2015), clean or noisy speech (Sriram et al., 2018), and invariant to differences between source languages in multi-lingual machine trans- lation (Xie et al., 2017). Elazar and Goldberg (2018) argue, however, that adversarial learning does not fully remove sensitive demographic traits from the data representations. This conclusion is based on the observation that a diagnostic clas- sifier trained over the supposedly debiased data representations could still predict gender, age and race above chance level in their experimental set- up. In general, diagnostic classifiers are trained on data representations to predict the protected de- mographic attributes in question as well as pos- sible, i.e., the classifier picks up on any correla- tions, strong or weak, between data representa- tions and the demographic classes. Correlations come in different flavours: PREVALENT: Certain features are indicative of gender in most contexts, e.g., the distribution of phrases like ‘as a mother’ or ‘as a guy’. SAMPLE-SPECIFIC: Some fea- tures are indicative of gender within a particular domain/sample, e.g. bling may correlate with a particular range of ages in a sample of reviews of accessories, and …

Content truncated to 500 words