Debiasing Pretrained Text Encoders by Paying Attention to Paying Attention

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9582 - 9602 December 7-11, 2022 ©2022 Association for Computational Linguistics Debiasing Pretrained Text Encoders by Paying Attention to Paying Attention Yacine Gaci1, Boualem Benatallah2,3, Fabio Casati4, Khalid Benabdeslem1 1 LIRIS - University of Lyon 1, France 2 Dublin City University, Ireland 3 UNSW, Sydney, Australia 4 ServiceNow, USA {yacine.gaci, khalid.benabdeslem}@univ-lyon1.fr {boualem.benatallah, fabio.casati}@gmail.com Abstract Natural Language Processing (NLP) models are found to exhibit discriminatory stereotypes across many social constructs, e.g. gender and race. In comparison to the progress made in re- ducing bias from static word embeddings, fair- ness in sentence-level text encoders received little consideration despite their wider appli- cability in contemporary NLP tasks. In this paper, we propose a debiasing method for pre- trained text encoders that both reduces social stereotypes, and inflicts next to no semantic damage. Unlike previous studies that directly manipulate the embeddings, we suggest to dive deeper into the operation of these encoders, and pay more attention to the way they pay attention to different social groups. We find that stereotypes are also encoded in the atten- tion layer. Then, we work on model debiasing by redistributing the attention scores of a text encoder such that it forgets any preference to historically advantaged groups, and attends to all social classes with the same intensity. Our experiments confirm that reducing bias from at- tention effectively mitigates it from the model’s text representations. 1 Introduction Natural Language Processing (NLP) is increasingly penetrating real-world processes such as recruit- ment (Hansen et al., 2015), legal systems (Dale, 2019), healthcare (Velupillai et al., 2018) and Web Search (Nalisnick et al., 2016). Part of this suc- cess is attributed to the underlying embedding layer which encodes sophisticated semantic representa- tions of language (Camacho-Collados and Pilehvar, 2018). However, this wide adoption has called into question the fairness aspect of modern text en- coders. Recent research exposed them for replicat- ing discriminatory social biases which may cause unintended and undesired model behaviors with respect to social groups (Bolukbasi et al., 2016; Caliskan et al., 2017; May et al., 2019). As an example, a resume filtering system built on top of such biased encoders display overly improper pref- erences towards male applicants (Dastin, 2018). Methods to debias text encoders have recently been proposed, ranging from Counterfactual Data Augmentation (CDA) (Webster et al., 2020), projec- tion on bias-free subspaces (Kaneko and Bollegala, 2021), contrastive learning (Cheng et al., 2020), adversarial attacks (Wang et al., 2021) or simply extending existing debiasing techniques from static word embeddings (Liang et al., 2020a; Bolukbasi et al., 2016; Manzini et al., 2019; Gaci et al., 2022a). However, these methods have shown mixed results, often failing to reduce the amount of bias to a satis- factory degree (Gonen and Goldberg, 2019; Blod- gett et al., 2020; Meade et al., 2021). We argue that part of this shortcoming owes to the fact that cur- rent debiasing techniques, by operating exclusively on the model’s embeddings, are not removing bias …

Content truncated to 500 words