Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1276 - 1295 July 10-15, 2022 ©2022 Association for Computational Linguistics Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models Yang Trista Cao∗1, Anna Sotnikova∗1 Hal Daumé III1,2 Rachel Rudinger1 Linda Zou1 1University of Maryland, College Park 2Microsoft Research {ycao95, asotniko, hal3, rudinger, lxzou}@umd.edu Abstract NLP models trained on text have been shown to reproduce human stereotypes, which can magnify harms to marginalized groups when systems are deployed at scale. We adapt the Agency-Belief-Communion (ABC) stereotype model of Koch et al. (2016) from social psy- chology as a framework for the systematic study and discovery of stereotypic group-trait associations in language models (LMs). We introduce the sensitivity test (SeT) for measur- ing stereotypical associations from language models. To evaluate SeT and other measures using the ABC model, we collect group-trait judgments from U.S.-based subjects to com- pare with English LM stereotypes. Finally, we extend this framework to measure LM stereo- typing of intersectional identities. 1 Introduction Stereotypes are abstract and over-generalized pic- tures in people’s minds that capture attributes about groups of people in the complex social world (Lippmann, 1965). They influence peo- ple’s thoughts and behaviors, and allow people to make predictions beyond their personal expe- rience or information given (Bruner et al., 1957; Wheeler and Petty, 2001). Stereotypes are also entwined with the production of prejudice, discrim- ination, and in-group favoritism (Stangor, 2014; Jackson, 2011). A long line of research in social psychology has established models of generic di- mensions that estimate people’s stereotypes of so- cial groups (Koch et al., 2016; Fiske et al., 2002, i.a.). We build on the Agency Beliefs Commu- nion (ABC) model, which measures stereotypes toward a social group with respect to 16 traits in three dimensions: Agency/Socioeconomic Suc- cess, Conservative–Progressive Beliefs, and Com- ∗ Equal contribution. Figure 1: Crowdsourced analysis of the social group “man” under the ABC model (Koch et al., 2016). Colors: purple=agency, red=belief, green=communion. munion (§2); an analysis of the group “man” across 32 traits (16 opposing dyads) is shown in Figure 1. Pre-trained language models (LMs) encode cor- relations between social groups and traits, like associating the group “Muslim” with the trait threatening, or “man” with confident (e.g., Ben- der et al., 2021; Nozza et al., 2021; Hovy and Yang, 2021). We conduct a systematic study of social stereotypes in contextualized English masked LMs, grounded in group-trait associations from the ABC model. To capture the group-trait associations in the LM, we first assess two previously proposed word association tests and also propose a new mea- surement: the sensitivity test (SeT) (§3). To evaluate the degree to which two LMs— BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019)—align with human stereotype judg- ments, we design a human study for collecting group-trait judgments (§4). We show that our mea- sure, SeT, best aligns with human judgements on group-trait associations and find that, in general, the association …

Content truncated to 500 words