SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1573–1596 May 2-6, 2023 ©2023 Association for Computational Linguistics SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models Haozhe An, Zongxia Li, Jieyu Zhao, Rachel Rudinger University of Maryland, College Park {haozhe, zli12321, jieyuz, rudinger}@umd.edu Abstract A common limitation of diagnostic tests for detecting social biases in NLP models is that they may only detect stereotypic associations that are pre-specified by the designer of the test. Since enumerating all possible prob- lematic associations is infeasible, it is likely these tests fail to detect biases that are present in a model but not pre-specified by the de- signer. To address this limitation, we pro- pose SODAPOP1 (SOcial bias Discovery from Answers about PeOPle), an approach for au- tomatic social bias discovery in social com- monsense question-answering. The SODAPOP pipeline generates modified instances from the Social IQa dataset (Sap et al., 2019b) by (1) substituting names associated with different de- mographic groups, and (2) generating many distractor answers from a masked language model. By using a social commonsense model to score the generated distractors, we are able to uncover the model’s stereotypic associations between demographic groups and an open set of words. We also test SODAPOP on debiased models and show the limitations of multiple state-of-the-art debiasing algorithms. 1 Introduction Researchers are increasingly aware of how NLP systems, especially widely used pre-trained lan- guage models like BERT (Devlin et al., 2019), cap- ture social biases. Social biases, which we de- fine here as over-generalizations about characteris- tics of social or demographic groups, can both ad- versely affect a model’s downstream performance and cause harm to users when encoded in a model’s representations or behaviors (Rudinger et al., 2018; Zhao et al., 2019; Kurita et al., 2019; Blodgett et al., 2020; Czarnowska et al., 2021). In this paper, we propose an approach to uncovering social biases in social commonsense reasoning models. It is partic- ularly important to examine social commonsense 1Code is available at https://github.com/haozhe-an/ SODAPOP. Figure 1: An example of a modified Social IQa MCQ sample. In an open-ended fashion, we generate distrac- tors (Answer A and B) that contain words uncovering model social biases when names associated with differ- ent demographic groups are inserted into the context and question. In this example, Answer A, with the pres- ence of “ruthless”, is a more successful distractor for African American female names, whereas Answer B is a more successful distractor for European American female names due to the word “funny”. Answer C is the correct answer choice from the Social IQa dataset. reasoning models because they are designed to rea- son about people and social interactions, and hence susceptible to stereotyped inferences. Biased infer- ences based on social group identities mentioned or alluded to in the input may cause representational harms to those group members. There have been consistent efforts to diagnose multiple types of social biases in NLP systems. Ex- isting methods for bias …

Content truncated to 500 words