Is the Lottery Fair? Evaluating Winning Tickets Across Demographics

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3214–3224 August 1–6, 2021. ©2021 Association for Computational Linguistics 3214 Is the Lottery Fair? Evaluating Winning Tickets Across Demographics Victor Petr´en Bach Hansen,1 2 Anders Søgaard1 1Department of Computer Science, University of Copenhagen, Denmark 2Topdanmark A/S, Denmark {victor.petren,soegaard}@di.ku.dk Abstract Recent studies have suggested that weight pruning, e.g. using lottery ticket extraction techniques (Frankle and Carbin, 2018), comes at the risk of compromising the group fairness of machine learning models (Paganini, 2020; Hooker et al., 2020), but to the best of our knowledge, no one has empirically evaluated this hypothesis at scale in the context of nat- ural language processing. We present experi- ments with two text classification datasets an- notated with demographic information: the Trustpilot Corpus (sentiment) and CivilCom- ments (toxicity). We evaluate the fairness of lottery ticket extraction through layer-wise and global weight pruning across three lan- guages and two tasks. Our results suggest that there is a small increase in group dispar- ity, which is most pronounced at high prun- ing rates and correlates with instability. The fairness of models trained with distribution- ally robust optimization objectives is some- times less sensitive to pruning, but results are not consistent. The code for our ex- periments is available at https://github. com/vpetren/fairness_lottery. 1 Introduction Heavily pruning deep neural network models is a way of reducing inference cost for resource- constrained environments, but does weight-pruning of deep neural networks increase their unfairness? Several recent papers suggest this (Paganini, 2020; Hooker et al., 2020), based on experiments from face and digit recognition, but does this also hold for natural language processing (NLP) models? Systematic biases may easily be exacerbated by pruning interventions in high-dimensional prob- lems because of feature swamping effects (Sutton et al., 2006). Overparameterized deep neural net- works generalize well, in part because they can hedge their bets and rely on multitudes of weak Figure 1: Fairness Sensitivity to Pruning (FSP): the gra- dient of the linear fit of (the logarithm of) the pruning ratio to min-max group-level disparity. We use this to quantify the sensitivity of Rawlsian min-max fairness to weight pruning across architectures, pruning strate- gies and datasets. evidence rather than the most prominent indepen- dent variables. Sparse models do not have that luxury and are therefore more sensitive to shifts (Globerson and Roweis, 2006; Søgaard, 2013). We introduce a fairness sensitivity to pruning metric that measures how Rawlsian min-max fair- ness across demographic groups changes with weight pruning. We estimate this sensitivity by taking the gradient of the linear fit of the loga- rithm of the pruning ratio to min-max group-level disparity. We show that across four datasets, fair- ness sensitivity to pruning is similar for layer-wise and global pruning strategies (Frankle and Carbin, 2018), as well as for text classifiers based on feed- forward and recurrent neural networks. Subse- quently, we consider the impact of a popular ro- bust optimization strategy designed to improve the fairness of classification models (Hashimoto et al., 2018; Sagawa et al., …

Content truncated to 500 words