Measuring Normative and Descriptive Biases in Language Models Using Census Data

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2242–2248 May 2-6, 2023 ©2023 Association for Computational Linguistics Measuring Normative and Descriptive Biases in Language Models Using Census Data Samia Touileb University of Bergen samia.touileb@uib.no Lilja Øvrelid University of Oslo liljao@ifi.uio.no Erik Velldal University of Oslo erikve@ifi.uio.no Abstract We investigate in this paper how distributions of occupations with respect to gender is re- flected in pre-trained language models. Such distributions are not always aligned to norma- tive ideals, nor do they necessarily reflect a de- scriptive assessment of reality. In this paper, we introduce an approach for measuring to what degree pre-trained language models are aligned to normative and descriptive occupational dis- tributions. To this end, we use official demo- graphic information about gender–occupation distributions provided by the national statistics agencies of France, Norway, United Kingdom, and the United States. We manually gener- ate template-based sentences combining gen- dered pronouns and nouns with occupations, and subsequently probe a selection of ten lan- guage models covering the English, French, and Norwegian languages. The scoring system we introduce in this work is language indepen- dent, and can be used on any combination of template-based sentences, occupations, and lan- guages. The approach could also be extended to other dimensions of national census data and other demographic variables. 1 Introduction Pre-trained language models (LMs) may contain various types of biases, and the field of NLP has seen a lot of work in recent years on attempting to identify, mitigate, and reduce these biases. Bi- ases can originate both from the unlabeled texts used for pre-training these LMs, and from texts and annotations used for tuning downstream clas- sifiers. LMs have become a cornerstone in most NLP model architectures, and the extent to which they reflect, amplify, and spread the biases present in their training data is still a problematic issue to be solved. Several efforts in this direction have focused on gender as a variable (Touileb and Nozza, 2022; Touileb et al., 2021; Ousidhoum et al., 2021; Nozza et al., 2021; Touileb et al., 2020; Saunders and Byrne, 2020; Bhaskaran and Bhallamudi, 2019; Cho et al., 2019; Prates et al., 2018), also in cor- relation with occupations (Borchers et al., 2022; Touileb et al., 2022; Bolukbasi et al., 2016). While there have been several efforts on exploring the ex- isting biases related to these demographic variables, most work approaches the task from a normative point of view (Blodgett, 2021), where equality be- tween the demographic distributions is prioritized. Although normativity in this aspect is crucial for certain applications, we argue that it is also interesting to explore the task from a descriptive perspective. This is especially interesting for oc- cupations, since a descriptive and realistic view of society already contains gender disparities. We pro- pose that national census data, in our case about gender-occupation distributions, can offer a reli- able ground truth against which model predictions can be compared. Moreover, we argue for taking both normative and …

Content truncated to 500 words