Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7237–7256 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7237 Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection Shauli Ravfogel1,2 Yanai Elazar1,2 Hila Gonen1 Michael Twiton3 Yoav Goldberg1,2 1Computer Science Department, Bar Ilan University 2Allen Institute for Artificial Intelligence 3Independent researcher {shauli.ravfogel, yanaiela, hilagnn, mtwito101, yoav.goldberg}@gmail.com Abstract The ability to control for the kinds of infor- mation encoded in neural representation has a variety of use cases, especially in light of the challenge of interpreting these mod- els. We present Iterative Null-space Projec- tion (INLP), a novel method for removing in- formation from neural representations. Our method is based on repeated training of lin- ear classifiers that predict a certain property we aim to remove, followed by projection of the representations on their null-space. By do- ing so, the classifiers become oblivious to that target property, making it hard to linearly sepa- rate the data according to it. While applicable for multiple uses, we evaluate our method on bias and fairness use-cases, and show that our method is able to mitigate bias in word em- beddings, as well as to increase fairness in a setting of multi-class classification. 1 Introduction What is encoded in vector representations of tex- tual data, and can we control it? Word embeddings, pre-trained language models, and more generally deep learning methods emerge as very effective techniques for text classification. Accordingly, they are increasingly being used for predictions in real-world situations. A large part of the suc- cess is due to the models’ ability to perform repre- sentation learning, coming up with effective fea- ture representations for the prediction task at hand. However, these learned representations, while ef- fective, are also notoriously opaque: we do not know what is encoded in them. Indeed, there is an emerging line of work on probing deep-learning derived representations for syntactic (Linzen et al., 2016; Hewitt and Manning, 2019; Goldberg, 2019), semantic (Tenney et al., 2019) and factual knowl- edge (Petroni et al., 2019). There is also evidence that they capture a lot of information regarding the Figure 1: t-SNE projection of GloVe vectors of the most gender-biased words after t=0, 3, 18, and 35 iter- ations of INLP. Words are colored according to being male-biased or female-biased. demographics of the author of the text (Blodgett et al., 2016; Elazar and Goldberg, 2018). What can we do in situations where we do not want our representations to encode certain kinds of information? For example, we may want a word representation that does not take tense into account, or that does not encode part-of-speech distinctions. We may want a classifier that judges the formality of the text, but which is also oblivious to the topic the text was taken from. Finally, and also our em- pirical focus in this work, this situation often arises when considering fairness and bias of language- based classification. We may not want our word- embeddings to encode gender stereotypes, …

Content truncated to 500 words