Gendered Mental Health Stigma in Masked Language Models

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2152 - 2170 December 7-11, 2022 ©2022 Association for Computational Linguistics Gendered Mental Health Stigma in Masked Language Models Inna Wanyin Lin1∗ Lucille Njoo1∗ Anjalie Field2 Ashish Sharma1 Katharina Reinecke1 Tim Althoff1 Yulia Tsvetkov1 1Paul G. Allen School of Computer Science & Engineering, University of Washington 2Stanford University {ilin, lnjoo}@cs.washington.edu Abstract Mental health stigma prevents many individ- uals from receiving the appropriate care, and social psychology studies have shown that men- tal health tends to be overlooked in men. In this work, we investigate gendered mental health stigma in masked language models. In doing so, we operationalize mental health stigma by de- veloping a framework grounded in psychology research: we use clinical psychology literature to curate prompts, then evaluate the models’ propensity to generate gendered words. We find that masked language models capture soci- etal stigma about gender in mental health: mod- els are consistently more likely to predict fe- male subjects than male in sentences about hav- ing a mental health condition (32% vs. 19%), and this disparity is exacerbated for sentences that indicate treatment-seeking behavior. Fur- thermore, we find that different models cap- ture dimensions of stigma differently for men and women, associating stereotypes like anger, blame, and pity more with women with mental health conditions than with men. In showing the complex nuances of models’ gendered men- tal health stigma, we demonstrate that context and overlapping dimensions of identity are im- portant considerations when assessing compu- tational models’ social biases. 1 Introduction Mental health issues are heavily stigmatized, pre- venting many individuals from seeking appropri- ate care (Sickel et al., 2014). In addition, social psychology studies have shown that this stigma manifests differently for different genders: mental illness is more visibly associated with women, but tends to be more harshly derided in men (Chatmon, 2020). This asymmetrical stigma constitutes harms towards both men and women, increasing the risks of under-diagnosis or over-diagnosis respectively. Since language is central to psychotherapy and peer support, NLP models have been increasingly ∗ Indicates equal contribution. Figure 1: We investigate masked language models’ bi- ases at the intersection of gender and mental health. Us- ing theoretically-motivated prompts about mental health conditions, we have models fill in the masked token, then examine the probabilities of generated words with gender associations. employed on mental health-related tasks (Chancel- lor and De Choudhury, 2020; Sharma et al., 2021, 2022; Zhang and Danescu-Niculescu-Mizil, 2020). Many approaches developed for these purposes rely on pretrained language models, thus running the risk of incorporating any pre-learned biases these models may contain (Straw and Callison-Burch, 2020). However, no prior research has examined how biases related to mental health stigma are rep- resented in language models. Understanding if and how pretrained language models encode mental health stigma is important for developing fair, re- sponsible mental health applications. To the best of our knowledge, our work is the first to opera- tionalize mental health stigma in NLP research and aim …

Content truncated to 500 words