Perturbation Augmentation for Fairer NLP

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9496 - 9521 December 7-11, 2022 ©2022 Association for Computational Linguistics Perturbation Augmentation for Fairer NLP Rebecca Qian† Candace Ross† Jude Fernandes† Eric Smith† Douwe Kiela‡, * Adina Williams†, * † Facebook AI Research; ‡ Hugging Face rebeccaqian,adinawilliams@fb.com Abstract Unwanted and often harmful social biases are becoming ever more salient in NLP research, affecting both models and datasets. In this work, we ask whether training on demograph- ically perturbed data leads to fairer language models. We collect a large dataset of human annotated text perturbations and train a neu- ral perturbation model, which we show out- performs heuristic alternatives. We find that (i) language models (LMs) pre-trained on de- mographically perturbed corpora are typically more fair, and (ii) LMs finetuned on perturbed GLUE datasets exhibit less demographic bias on downstream tasks, and (iii) fairness improve- ments do not come at the expense of perfor- mance on downstream tasks. Lastly, we discuss outstanding questions about how best to eval- uate the (un)fairness of large language mod- els. We hope that this exploration of neural demographic perturbation will help drive more improvement towards fairer NLP. 1 Introduction There is increasing evidence that models can instantiate social biases (Buolamwini and Gebru, 2018; Stock and Cissé, 2018; Fan et al., 2019; Merullo et al., 2019; Prates et al., 2020), often replicating or amplifying harmful statistical associations in their training data (Caliskan et al., 2017; Chang et al., 2019). Training models on data with representational issues can lead to unfair or poor treatment of particular demographic groups (Baro- cas et al., 2017; Mehrabi et al., 2021), a problem that is particularly egregious for historically marginalized groups, including people of color (Field et al., 2021), and women (Hendricks et al., 2018). As NLP moves towards training models on ever larger data samples (Ka- plan et al., 2020), such data-related risks may grow (Ben- der et al., 2021). In this work, we explore the efficacy of a dataset alter- ation technique that rewrites demographic references in text, such as changing “women like shopping” to “men like shopping”. Similar demographic perturbation ap- proaches have been fruitfully used to measure and often lessen the severity of social bias in text data (Hall Maud- slay et al., 2019; Prabhakaran et al., 2019; Zmigrod et al., 2019; Dinan et al., 2020a,b; Webster et al., 2020; Ma et al., 2021; Smith and Williams, 2021; Renduch- intala and Williams, 2022; Emmery et al., 2022). Most approaches for perturbing demographic references, how- ever, rely on rule-based systems, which unfortunately tend to be rigid and error prone, resulting in noisy and unnatural perturbations (see Section 4). While some have suggested that a neural demographic perturbation model may generate higher quality text rewrites, there are currently no annotated datasets large enough for training neural models (Sun et al., 2021). In this work, we collect the first large-scale dataset of 98K human-generated demographic text perturbations, the Perturbation Augmentation NLP DAtaset (PANDA). We use PANDA …

Content truncated to 500 words