SocioProbe: What, When, and Where Language Models Learn about Sociodemographics

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7901 - 7918 December 7-11, 2022 ©2022 Association for Computational Linguistics SocioProbe: What, When, and Where Language Models Learn about Sociodemographics Anne Lauscher1, Federico Bianchi2, Samuel Bowman3, and Dirk Hovy4 1Data Science Group, University of Hamburg, Germany 2StanfordNLP, Stanford University, CA, USA 3New York University, NY, USA 4MilaNLP, Bocconi University, Milan, Italy anne.lauscher@uni-hamburg.de, fede@stanford.edu, bowman@nyu.edu, dirk.hovy@unibocconi.it Abstract Pre-trained language models (PLMs) have out- performed other NLP models on a wide range of tasks. Opting for a more thorough under- standing of their capabilities and inner work- ings, researchers have established the extend to which they capture lower-level knowledge like grammaticality, and mid-level semantic knowledge like factual understanding. How- ever, there is still little understanding of their knowledge of higher-level aspects of language. In particular, despite the importance of sociode- mographic aspects in shaping our language, the questions of whether, where, and how PLMs encode these aspects, e.g., gender or age, is still unexplored. We address this research gap by probing the sociodemographic knowl- edge of different single-GPU PLMs on multi- ple English data sets via traditional classifier probing and information-theoretic minimum description length probing. Our results show that PLMs do encode these sociodemographics, and that this knowledge is sometimes spread across the layers of some of the tested PLMs. We further conduct a multilingual analysis and investigate the effect of supplementary train- ing to further explore to what extent, where, and with what amount of pre-training data the knowledge is encoded. Our overall results indi- cate that sociodemographic knowledge is still a major challenge for NLP. PLMs require large amounts of pre-training data to acquire the knowledge and models that excel in general language understanding do not seem to own more knowledge about these aspects. 1 Introduction When talking to somebody, we consciously choose how to represent ourselves, and we have a mental model of who our conversational partner is. At the same time, our language is littered with sub- conscious clues about our sociodemographic back- ground that we cannot control (e.g., our age, edu- cation, regional origin, social class, etc). People use this information as an integral part of language, to better reach their audience, and to understand what they are saying (e.g., Trudgill, 2000). In other words, we use sociodemographic knowledge to decide what to say (are we talking to a child or an adult, do I want to sound smart or relatable?) But do pre-trained language models (PLMs) have knowledge about sociodemographics? Over the last years, PLMs like BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019) have achieved superior performance on a wide range of downstream tasks (e.g., Wang et al., 2018, 2019, inter alia). Accordingly, they have become the de facto standard for most NLP tasks. Conse- quently, many researchers have tried to shed light on PLMs’ inner workings (cf. “Bertology”; Ten- ney et al., 2019; Rogers et al., 2020). They have systematically probed the models’ capabilities to unveil which …

Content truncated to 500 words