“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9180 - 9211 December 7-11, 2022 ©2022 Association for Computational Linguistics “I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset Eric Michael Smith Melissa Hall Melanie Kambadur Eleonora Presani Meta AIAdina Williams {ems,melissahall,mkambadur,epresani,adinawilliams}@fb.com Abstract As language models grow in popularity, it be- comes increasingly important to clearly mea- sure all possible markers of demographic iden- tity in order to avoid perpetuating existing so- cietal harms. Many datasets for measuring bias currently exist, but they are restricted in their coverage of demographic axes and are commonly used with preset bias tests that pre- suppose which types of biases models can ex- hibit. In this work, we present a new, more inclusive bias measurement dataset, HOLIS- TICBIAS, which includes nearly 600 descriptor terms across 13 different demographic axes. HOLISTICBIAS was assembled in a partici- patory process including experts and commu- nity members with lived experience of these terms. These descriptors combine with a set of bias measurement templates to produce over 450,000 unique sentence prompts, which we use to explore, identify, and reduce novel forms of bias in several generative models. We demonstrate that HOLISTICBIAS is effective at measuring previously undetectable biases in token likelihoods from language models, as well as in an offensiveness classifier. We will invite additions and amendments to the dataset, which we hope will serve as a basis for more easy-to-use and standardized methods for eval- uating bias in NLP models. 1 Introduction In recent years, there has been a series of works aiming to measure social biases or other unwanted behaviors in NLP. In particular, many works fo- cus on generative models (Dinan et al., 2020a,b; Xu et al., 2021b; Kirk et al., 2021; Sheng et al., 2021b; Nozza et al., 2021; Renduchintala et al., 2021; Baheti et al., 2021; Perez et al., 2022), which are well known to pose unique challenges for au- tomatic evaluation (Lowe et al., 2017; Howcroft et al., 2020; Celikyilmaz et al., 2021). For models that generate, a common way to surface bias is to input prompts containing demo- Ability Axis Age Body type Bucket auditory ... Descriptor “Deaf” “hard-of-hearing” “hearing-impaired” … mobility “paraplegic” “paralyzed” “quadriplegic” … Characteristics Sexual orientation Cultural Gender/sex Nationality Nonce Political Race/ethnicity Socioeconomic Religion asian ... “Asian” “Asian-American” “Desi” … Figure 1: The HOLISTICBIAS dataset has 13 different demographic axes, plus further divisions into buckets and nearly 600 individual descriptor terms. graphic information, and then analyze whether the models output socially biased text. Such prompts are generally derived either from crowdsourcing (Nadeem et al., 2021; Nangia et al., 2021) or from slotting a set of terms into templates (Kurita et al., 2019; May et al., 2019; Sheng et al., 2019; Webster et al., 2020). However, whenever a method se- lects particular terms or templates for prompts, and groups them under particular demographic head- ings, it implicitly adopts a taxonomy which can include, or exclude, particular groups of people …

Content truncated to 500 words