Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology

Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology Ran Zmigrod1 Sabrina J. Mielke2 Hanna Wallach3 Ryan Cotterell1 1 University of Cambridge 2 Johns Hopkins University 3 Microsoft Research rz279@cam.ac.uk sjmielke@jhu.edu wallach@microsoft.com rdc42@cam.ac.uk Abstract Gender stereotypes are manifest in most of the world’s languages and are consequently propa- gated or amplified by NLP systems. Although research has focused on mitigating gender stereotypes in English, the approaches that are commonly employed produce ungrammatical sentences in morphologically rich languages. We present a novel approach for convert- ing between masculine-inflected and feminine- inflected sentences in such languages. For Spanish and Hebrew, our approach achieves F1 scores of 82% and 73% at the level of tags and accuracies of 90% and 87% at the level of forms. By evaluating our approach using four different languages, we show that, on average, it reduces gender stereotyping by a factor of 2.5 without any sacrifice to grammaticality. 1 Introduction One of the biggest challenges faced by modern natural language processing (NLP) systems is the inadvertent replication or amplification of societal biases. This is because NLP systems depend on lan- guage corpora, which are inherently “not objective; they are creations of human design” (Crawford, 2013). One type of societal bias that has received considerable attention from the NLP community is gender stereotyping (Garg et al., 2017; Rudinger et al., 2017; Sutton et al., 2018). Gender stereo- types can manifest in language in overt ways. For example, the sentence he is an engineer is more likely to appear in a corpus than she is an engineer due to the current gender disparity in engineering. Consequently, any NLP system that is trained such a corpus will likely learn to associate engineer with men, but not with women (De-Arteaga et al., 2019). To date, the NLP community has focused pri- marily on approaches for detecting and mitigating gender stereotypes in English (Bolukbasi et al., 2016; Dixon et al., 2018; Zhao et al., 2017). Yet, gender stereotypes also exist in other languages Los ingenieros son expertos Analysis El ingeniero ser experto DET NOUN VERB ADJ [MSC; PL] [MSC; PL] [IN; PR; PL] [MSC; PL] Intervention El ingeniera ser experto DET NOUN VERB ADJ [MSC; PL] [FEM; PL] [IN; PR; PL] [MSC; PL] Inference El ingeniera ser experto DET NOUN VERB ADJ [FEM; PL] [FEM; PL] [IN; PR; PL] [FEM; PL] Reinflection Las ingenieras son expertas Figure 1: Transformation of Los ingenieros son expertos (i.e., The male engineers are skilled) to Las ingenieras son expertas (i.e., The female engineers are skilled). We extract the properties of each word in the sentence. We then fix a noun and its tags and infer the manner in which the remaining tags must be updated. Finally, we reinflect the lemmata to their new forms. because they are a function of society, not of gram- mar. Moreover, because English does not mark grammatical gender, approaches developed for En- glish are not transferable to morphologically rich languages that exhibit gender agreement (Corbett, 1991). In these languages, …

Content truncated to 500 words