Analyzing Stereotypes in Generative Text Inference Tasks

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4052–4065 August 1–6, 2021. ©2021 Association for Computational Linguistics 4052 Analyzing Stereotypes in Generative Text Inference Tasks Anna Sotnikova∗ University of Maryland asotniko@umd.edu Yang Trista Cao∗ University of Maryland ycao95@umd.edu Hal Daum´e III University of Maryland Microsoft Research me@hal3.name Rachel Rudinger University of Maryland rudinger@umd.edu Abstract Stereotypes are inferences drawn about people based on their demographic attributes, which may result in harms to users when a system is deployed. In generative language-inference tasks, given a premise, a model produces plausible hypotheses that follow either logi- cally (natural language inference) or common- sensically (commonsense inference). Such tasks are therefore a fruitful setting in which to explore the degree to which NLP systems encode stereotypes. In our work, we study how stereotypes manifest when the potential targets of stereotypes are situated in real-life, neutral contexts. We collect human judgments on the presence of stereotypes in generated inferences, and compare how perceptions of stereotypes vary due to annotator positionality. 1 Introduction Social categories refer to collections of people with shared traits; stereotypes—cognitive structures that associate categories (e.g., man, Black, poor, professor) with both roles (e.g., doctor) and traits (e.g., absent-minded)—are central to how people construe social meaning (Levon, 2014; Macrae and Bodenhausen, 2001; Greenwald et al., 1998). Social psychology has studied how stereotypes, as a cognitive process, are entwined with the production of human affects of prejudice and in-group favoritism, as well as behaviors like discrimination (Stangor, 2014; Jackson, 2011). Linguistic anthropology and sociolinguistic studies argue that language—as the predominant way of naming categories and transmitting knowledge—is the only (or at least the primary) mechanism by which social stereotypes are shared as part of cultural knowledge (Fishman, 1956; Stangor and Schaller, 2012; Maass and Arcuri, 1996). ∗⋆ Equal contribution premise A Cuban person needs help. ↰ hypothesis Then PERSONX gets a job. Question Annotator 1 Annotator 2 correct? yes yes plausible? yes yes identity? maybe yes no situation? maybe no not sure sentiment? maybe negative positive stereotype? yes no description? problems with jobs n/a Table 1: Annotation example; the hypothesis is auto- matically generated from the premise. Both annotators found the hypothesis grammatically correct and plau- sible. One annotator viewed this hypothesis as nega- tive stereotypical towards Cuban people, assuming that they have problems with jobs. The other annotator had the opposite opinion. Annotators differ in their back- grounds and social groups they belong to. In this paper, we study ways in which categories implicate inferences around stereotypical roles and traits computationally.1 Approaching stereotyping through the lens of inference allows us to focus on what models learn as implications rather than simply associations (e.g., that lexical semantics models typically find antonyms like “hot” and “cold” to be highly related). Specifically, we train models for English textual inference—including both logical- (NLI) and commonsense-inference (CI)—and investigate how stereotypes are repro- duced by these models. The models we train generate hypothesis text given a fixed premise text (e.g., “PERSONX lights up candles”, where PERSONX is substituted …

Content truncated to 500 words