Counterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4716 - 4726 July 10-15, 2022 ©2022 Association for Computational Linguistics Counterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection Indira Sen1 Mattia Samory1 Claudia Wagner1,2 Isabelle Augenstein3 1GESIS – Leibniz Institute for the Social Sciences 3University of Copenhagen2RWTH Aachen University {indira.sen, mattia.samory, claudia.wagner}gesis.org; augenstein@di.ku.dk Abstract Counterfactually Augmented Data (CAD) aims to improve out-of-domain generalizabil- ity, an indicator of model robustness. The improvement is credited to promoting core features of the construct over spurious arti- facts that happen to correlate with it. Yet, over-relying on core features may lead to un- intended model bias. Especially, construct- driven CAD—perturbations of core features— may induce models to ignore the context in which core features are used. Here, we test models for sexism and hate speech detection on challenging data: non-hateful and non- sexist usage of identity and gendered terms. On these hard cases, models trained on CAD, especially construct-driven CAD, show higher false positive rates than models trained on the original, unperturbed data. Using a diverse set of CAD—construct-driven and construct- agnostic—reduces such unintended bias. 1 Introduction As fully or semi-automated models are increasingly used for platform governance (Gorwa, 2019; Gille- spie, 2018; Nakov et al., 2021) there are several questions about their performance and the implica- tions of model errors (Gorwa et al., 2020; Gillespie, 2020; Roberts, 2019). Language technologies un- derpinning these content moderation strategies, es- pecially models for detecting problematic content like hate speech and sexism, need to be designed to ensure several complex desiderata, including robustness across domains of application as well as low misclassification rates. Indeed, misclassi- fications can have a range of repercussions from allowing problematic content to proliferate to sanc- tioning users who did nothing wrong, often minori- ties and activists (Gray and Stein, 2021; Haimson et al., 2021). Such misclassifications are a threat to model robustness and non-robust models can cause a great deal of collateral damage. To facilitate model robustness, several solu- tions encompass improving training data for these models (Dinan et al., 2019; Vidgen et al., 2021), such as by training them on counterfactually aug- mented data (CAD). CAD, also called contrast sets (Gardner et al., 2020; Atanasova et al., 2022), are obtained by making minimal changes to ex- isting datapoints to flip their label for a particu- lar NLP task (Kaushik et al., 2019; Samory et al., 2021). Previous research has established that train- ing on CAD increases out-of-domain generalizabil- ity (Kaushik et al., 2019; Samory et al., 2021; Sen et al., 2021). Sen et al. (2021) explores charac- teristics of effective counterfactuals, finding that models trained on construct-driven CAD, or CAD obtained by directly perturbing manifestations of the construct, e.g., gendered words for sexism, lead to higher out-of-domain generalizability. Previous research also notes that gains from training on CAD can be attributed to learning more core features, rather than dataset artifacts (Kaushik et al., 2019; …

Content truncated to 500 words