SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9851–9870 July 9-14, 2023 ©2023 Association for Computational Linguistics SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models Akshita Jha∗ Virginia Tech akshitajha@vt.edu Aida Davani Google Research aidamd@google.com Chandan K. Reddy Virginia Tech reddy@cs.vt.edu Shachi Dave Google Research shachi@google.com Vinodkumar Prabhakaran Google Research vinodkpg@google.com Sunipa Dev Google Research sunipadev@google.com Abstract Stereotype benchmark datasets are crucial to detect and mitigate social stereotypes about groups of people in NLP models. However, existing datasets are limited in size and cov- erage, and are largely restricted to stereotypes prevalent in the Western society. This is es- pecially problematic as language technologies gain hold across the globe. To address this gap, we present SeeGULL, a broad-coverage stereotype dataset, built by utilizing genera- tive capabilities of large language models such as PaLM, and GPT-3, and leveraging a glob- ally diverse rater pool to validate the preva- lence of those stereotypes in society. SeeG- ULL is in English, and contains stereotypes about identity groups spanning 178 countries across 8 different geo-political regions across 6 continents, as well as state-level identities within the US and India. We also include fine-grained offensiveness scores for different stereotypes and demonstrate their global dis- parities. Furthermore, we include comparative annotations about the same groups by anno- tators living in the region vs. those that are based in North America, and demonstrate that within-region stereotypes about groups differ from those prevalent in North America. CONTENT WARNING: This paper contains stereotype examples that may be offensive. 1 Introduction Language technologies have recently seen impres- sive gains in their capabilities and potential down- stream applications, mostly aided by advancements in large language models (LLMs) trained on web data (Bommasani et al., 2021). However, there is also increasing evidence that these technologies may reflect and propagate undesirable societal bi- ases and stereotypes (Kurita et al., 2019; Sheng ∗Work done while at Google Research et al., 2019; Khashabi et al., 2020; Liu et al., 2019; He et al., 2020). Stereotypes are generalized beliefs about categories of people,1 and are often reflected in data as statistical associations, which the lan- guage models rely on to associate concepts. For in- stance, Parrish et al. (2022) demonstrate that LLM- based question-answer models rely on stereotypes to answer questions in under-informative contexts. Not all statistical associations learned from data about a subgroup are stereotypes; for instance, data may associate women with both breast cancer and nursing as a profession, but only the latter asso- ciation is a commonly held stereotype (Wilbourn and Kee, 2010). Recent work has built stereotype benchmark datasets (e.g., StereoSet (Nadeem et al., 2021), CrowS-Pairs (Nangia et al., 2020)) aimed to detect such stereotypes in NLP model predic- tions. While these datasets have been instrumental in demonstrating that language models may rein- force stereotypes, they have several key limitations. First, they are limited in their size and coverage, especially for subgroups across the globe. Second, they are curated exclusively …

Content truncated to 500 words