Learning Fair Representations via Rate-Distortion Maximization

Learning Fair Representations via Rate-Distortion Maximization Somnath Basu Roy Chowdhury and Snigdha Chaturvedi UNC Chapel Hill, USA {somnath, snigdha}@cs.unc.edu Abstract Text representations learned by machine learning models often encode undesirable de- mographic information of the user. Predictive models based on these representations can rely on such information, resulting in biased deci- sions. We present a novel debiasing technique, Fairness-aware Rate Maximization (FaRM), that removes protected information by making representations of instances belonging to the same protected attribute class uncorrelated, us- ing the rate-distortion function. FaRM is able to debias representations with or without a tar- get task at hand. FaRM can also be adapted to remove information about multiple protected attributes simultaneously. Empirical evalua- tions show that FaRM achieves state-of-the-art performance on several datasets, and learned representations leak significantly less pro- tected attribute information against an attack by a non-linear probing network. 1 Introduction Democratization of machine learning has led to deployment of predictive models for critical appli- cations like credit approval (Ghailan et al., 2016) and college application reviewing (Basu et al., 2019). Therefore, it is important to ensure that decisions made by these models are fair towards different demographic groups (Mehrabi et al., 2021). Fairness can be achieved by ensuring that the demographic information does not get en- coded in the representations used by these mod- els (Blodgett et al., 2016; Elazar and Goldberg, 2018; Elazar et al., 2021). However, controlling demographic informa- tion encoded in a model’s representations is a challenging task for textual data. This is because natural language text is highly indicative of an au- thor’s demographic attributes even when it is not explicitly mentioned (Koppel et al., 2002; Burger et al., 2011; Nguyen et al., 2013; Verhoeven and Daelemans, 2014; Weren et al., 2014; Rangel et al., 2016; Verhoeven et al., 2016; Blodgett et al., 2016). In this work, we debias information about a protected attribute (e.g., gender, race) from textual data representations. Previous debiasing methods (Bolukbasi et al., 2016; Ravfogel et al., 2020) project representations in a subspace that does not reveal protected attribute information. These methods are only able to guard protected attributes against an attack by a linear function (Ravfogel et al., 2020). Other methods (Xie et al., 2017; Basu Roy Chowdhury et al., 2021) adversarially remove protected information while retaining information about a target attribute. However, they are difficult to train (Elazar and Goldberg, 2018) and require a target task at hand. We present a novel debiasing technique, Fairness-aware Rate Maximization (FaRM), that removes demographic information by control- ling the rate-distortion function of the learned representations. Intuitively, in order to remove information about a protected attribute from a set of representations, we want the representa- tions from the same protected attribute class to be uncorrelated to each other. We achieve this by maximizing the number of bits (rate-distortion) required to encode representations with the same protected attribute. Figure 1 illustrates the pro- cess. The representations are shown as points in a two-dimensional feature space, color-coded ac- cording to their protected …

Content truncated to 500 words