Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1665–1670 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 1665 Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation Samhita Honnavalli∗1, Aesha Parekh∗1, Lily Ou∗1, Sophie Groenwold∗1, Sharon Levy1, Vicente Ordonez2, William Yang Wang1 1Department of Computer Science, University of California Santa Barbara 2Department of Computer Science, Rice University {shonnavalli, aeshaparekh, lilyou, sophiegroenwold}@ucsb.edu {sharonlevy,william}@cs.ucsb.edu vicenteor@rice.edu Abstract Women are often perceived as junior to their male counterparts, even within the same job titles. While there has been significant progress in the evaluation of gender bias in natural language processing (NLP), existing studies seldom investigate how biases toward gender groups change when compounded with other societal biases. In this work, we investigate how seniority impacts the degree of gender bias exhibited in pretrained neural generation models by introducing a novel framework for probing compound bias. We contribute a benchmark robustness-testing dataset spanning two domains, U.S. senatorship and professorship, created using a distant-supervision method. Our dataset includes human-written text with underlying ground truth and paired counterfactuals. We then examine GPT-2 perplexity and the frequency of gendered language in generated text. Our results show that GPT-2 amplifies bias by considering women as junior and men as senior more often than the ground truth in both domains. These results suggest that NLP applications built using GPT-2 may harm women in professional capacities. Keywords: gender-seniority bias, natural language generation, dataset creation 1. Introduction Propagation of societal biases is a growing issue in mainstream natural language generation (NLG) mod- els. Downstream applications of these models, such as machine translation (Koehn, 2009), dialogue genera- tion (Serban et al., 2016), and story generation (Yao et al., 2019) risk reinforcing societal stereotypes. One of the most well-known types of societal bias in natural language processing (NLP) is gender bias (Sun et al., 2019; Zhao et al., 2019; Bolukbasi et al., 2016; Rudinger et al., 2018). Previous work has revealed gender bias in coreference systems using an evalua- tion corpus that links gendered entities to various oc- cupations (Zhao et al., 2018). Similarly, Kurita et al. (2019) quantifies gender bias using probabilities that BERT (Devlin et al., 2019) assigns to sentences that as- sociate gendered words with career-related words. Al- though the impact of gender bias on NLP tasks has been consistently identified and measured (Zhao et al., 2017; Bordia and Bowman, 2019), we hypothesize that it does not occur in isolation. In this paper, we view bias through a multidimensional lens by studying com- pound gender-seniority bias. Due to gender stereotypes, traits typically associated with high-seniority positions, such as leaders in a given field, are more often attributed to men than to women (Eagly and Karau, 2002; Heilman, 2012). Conse- quently, natural language generation (NLG) models may be perpetuating biased information about gen- dered entities with respect to their perceived senior- ity level. We have seen how bias in NLP has dispro- ∗ Equal contribution ORIGINAL Our junior Senator Shelley Moore Capito …

Content truncated to 500 words