VLStereoSet: A Study of Stereotypical Bias in Pre-trained Vision-Language Models

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 527–538 November 20–23, 2022. ©2022 Association for Computational Linguistics 527 VLStereoSet: A Study of Stereotypical Bias in Pre-trained Vision-Language Models Kankan Zhou, Eason Lai, Jing Jiang School of Computing and Information Systems, Singapore Management University kkzhou.2020@smu.edu.sg, yblai@smu.edu.sg, jingjiang@smu.edu.sg Abstract Warning: This paper may contain images and texts with uncomfortable content. In this paper we study how to measure stereo- typical bias in pre-trained vision-language mod- els. We leverage a recently released text-only dataset, StereoSet, which covers a wide range of stereotypical bias, and extend it into a vision- language probing dataset called VLStereoSet to measure stereotypical bias in vision-language models. We analyze the differences between text and image and propose a probing task that detects bias by evaluating a model’s tendency to pick stereotypical statements as captions for anti-stereotypical images. We further de- fine several metrics to measure both a vision- language model’s overall stereotypical bias and its intra-modal and inter-modal bias. Experi- ments on six representative pre-trained vision- language models demonstrate that stereotypi- cal biases clearly exist in most of these models and across all four bias categories, with gender bias slightly more evident. Further analysis us- ing gender bias data and two vision-language models also suggest that both intra-modal and inter-modal bias exist. 1 Introduction Recently there has been much interest in adapting foundation models such as ALBERT (Lan et al., 2020),RoBERTa (Liu et al., 2020), T5 (Raffel et al., 2020), GPT-3 (Brown et al., 2020) and CLIP (Rad- ford et al., 2021) for different downstream tasks. These models demonstrate powerful transfer capa- bilities largely because they have acquired the rich body of knowledge contained in their pre-training data. However, their pre-training data may also contain social biases and stereotypes, especially when the data are crawled from the internet with- out cleaning. As a result, pre-trained models may “inherit” these biases and stereotypes, affecting the fairness of systems derived from these foundation models for downstream tasks. Figure 1: An image and its three candidate captions in our VLStereoSet. Sister represents a target social group and caring, rude and hi are three attributes. Previous work mainly focused on measuring biases and stereotypes in a single modality. For example, in NLP, people studied social biases in word embeddings (e.g., Bolukbasi et al., 2016, Zhao et al., 2018b) and language models (e.g., Nadeem et al., 2021,Abid et al., 2021), and in com- puter vision, people studied social biases in unsu- pervised vision models (e.g., Steed and Caliskan, 2021). However, there has been little work to under- stand social biases in multi-modal or cross-modal settings. In particular, although there has been fast progress recently in developing large-scale pre-trained vision-language models (e.g., Li et al., 2021; Radford et al., 2021; Singh et al., 2022), be- cause these models are relatively new, little work has been done to understand biases and stereotypes in them. …

Content truncated to 500 words