Gold Doesn’t Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Information

Gold Doesn’t Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Information Shun Shao Yftah Ziser Shay B. Cohen Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh, EH8 9AB s.shao-11@sms.ed.ac.uk yftah.ziser@ed.ac.uk scohen@inf.ed.ac.uk Abstract We describe a simple and effective method (Spectral Attribute removaL; SAL) to remove private or guarded information from neural rep- resentations. Our method uses matrix decom- position to project the input representations into directions with reduced covariance with the guarded information rather than maximal covariance as factorization methods normally use. We begin with linear information removal and proceed to generalize our algorithm to the case of nonlinear information removal using kernels. Our experiments demonstrate that our algorithm retains better main task performance after removing the guarded information com- pared to previous work. In addition, our exper- iments demonstrate that we need a relatively small amount of guarded attribute data to re- move information about these attributes, which lowers the exposure to sensitive data and is more suitable for low-resource scenarios.1 1 Introduction Natural language processing (NLP) models cur- rently play a critical role in decision-supporting systems. Their predictions are often affected by undesirable biases encoded in real-world data they are trained on. Making sensitive predictions based on irrelevant input attributes such as gender, race, or religion (protected or guarded attributes) im- pacts user trust and the practical broad utility of NLP methods. In recent years, representation learning ap- proaches have become the mainstay of input en- coding in NLP. While representation learning has yielded state-of-the-art results in many NLP tasks, controlling or inspecting the information encoded in these representations is hard. Thus, using rule- based methods to remove unwanted information from such representations is often not feasible. In 1Code is available at https://github.com/ jasonshaoshun/SAL. accountant architect attorney chiropractor comedian composer dentist dietitian dj filmmaker interior designer journalist model nurse painter paralegal pastor personal trainer photographer physician poet professor psychologist rapper software engineer surgeon teacher yoga teacher professions 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 intensity (male) (female) (inter-cluster) Figure 1: The ratio ρ between the average t- SNE similarity of representations between two gen- der clusters c1, c2 (sim(c1, c2)) for each profession: ρ = � after SAL sim(c1, c2) � before SAL sim(c1, c2) � Three values of ρ are computed, intra-cluster: (1). c1 = c2 = male; (2) c1 = c2 = female; and inter- cluster: (3) c1 = male, c2 = female. The ratios in the inter-cluster case are smaller than 1, and larger than 1 for the intra-cluster case. the context of protected attributes, Bolukbasi et al. (2016) showed that word embeddings trained on the Google News corpus encode gender stereotypes. Later, Manzini et al. (2019) expanded this work and showed that word embeddings trained on the Red- dit L2 corpus (Rabinovich et al., 2018) encode race and religion biases. We propose a simple yet effective technique to remove protected attribute information from neu- ral representations. Our method, dubbed SAL for Spectral …

Content truncated to 500 words