Counter-GAP: Counterfactual Bias Evaluation through Gendered Ambiguous Pronouns

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3761–3773 May 2-6, 2023 ©2023 Association for Computational Linguistics Counter-GAP: Counterfactual Bias Evaluation through Gendered Ambiguous Pronouns Zhongbin Xie1, Vid Kocijan2, Thomas Lukasiewicz3,1, Oana-Maria Camburu4 1 University of Oxford 2 Kumo.ai 3 TU Wien 4 University College London zhongbin.xie@cs.ox.ac.uk, vid@kumo.ai, thomas.lukasiewicz@tuwien.ac.at, o.camburu@ucl.ac.uk Abstract Bias-measuring datasets play a critical role in detecting biased behavior of language models and in evaluating progress of bias mitigation methods. In this work, we focus on evaluat- ing gender bias through coreference resolution, where previous datasets are either hand-crafted or fail to reliably measure an explicitly defined bias. To overcome these shortcomings, we pro- pose a novel method to collect diverse, natu- ral, and minimally distant text pairs via coun- terfactual generation, and construct Counter- GAP, an annotated dataset consisting of 4008 instances grouped into 1002 quadruples. We further identify a bias cancellation problem in previous group-level metrics on Counter-GAP, and propose to use the difference between in- consistency across genders and within genders to measure bias at a quadruple level. Our results show that four pre-trained language models are significantly more inconsistent across different gender groups than within each group, and that a name-based counterfactual data augmentation method is more effective to mitigate such bias than an anonymization-based method. 1 Introduction It is a common practice to train state-of-the-art nat- ural language processing (NLP) models by unsuper- vised pre-training and supervised fine-tuning (e.g., Devlin et al., 2019; Joshi et al., 2020), both of which rely heavily on large corpora of real-world text. However, these corpora often reflect societal stereotypes and may lead to models exhibiting bi- ased behaviors (Bender et al., 2021). Hence, much research effort has been put to reveal and mitigate unintended biases (Meade et al., 2022). While early work focuses on detecting and mit- igating gender bias in the space of word embed- dings (e.g., Bolukbasi et al., 2016), recent ap- proaches turn to design bias-measuring datasets on specific NLP tasks (Nangia et al., 2020; Nadeem et al., 2021; Barikeri et al., 2021). In this work, we focus on gender bias in coreference resolution and adopt a kind of representational harm (Blodgett et al., 2020) to define gender fairness: a gender- neutral model should rely on the semantic infor- mation, rather than on the gender information con- tained in the texts, to make predictions. Otherwise, a model should be considered gender-biased. In line with this definition, WinoBias (Zhao et al., 2018) and WinoGender (Rudinger et al., 2018) leverage pairs of minimally distant sentences, i.e., two sentences that contain the same semantic in- formation but different gender information, to mea- sure models’ performance difference in resolving pronouns of different genders under the same con- text. This minimally distant setting enables us to isolate the influence of gender information on model predictions. A limitation of WinoBias and WinoGender is that they are made up of hand-crafted sentences, which prevents us from measuring gender bias in the more diverse …

Content truncated to 500 words