Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1995–2008 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1995 Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search Jialu Wang, Yang Liu, Xin Eric Wang Department of Computer Science and Engineering University of California, Santa Cruz {faldict,yangliu,xwang366}@ucsc.edu Abstract Internet search affects people’s cognition of the world, so mitigating biases in search re- sults and learning fair models is imperative for social good. We study a unique gender bias in image search in this work: the search im- ages are often gender-imbalanced for gender- neutral natural language queries. We diag- nose two typical image search models, the spe- cialized model trained on in-domain datasets and the generalized representation model pre- trained on massive image and text data across the internet. Both models suffer from severe gender bias. Therefore, we introduce two novel debiasing approaches: an in-processing fair sampling method to address the gender im- balance issue for training models, and a post- processing feature clipping method base on mutual information to debias multimodal rep- resentations of pre-trained models. Extensive experiments on MS-COCO (Lin et al., 2014) and Flickr30K (Young et al., 2014) bench- marks show that our methods significantly re- duce the gender bias in image search models. 1 Introduction Internet information is shaping people’s minds. The algorithmic processes behind modern search engines, with extensive use of machine learning, have great power to determine users’ access to in- formation (Eslami et al., 2015). These information systems are biased when results are systematically slanted in unfair discrimination against protected groups (Friedman and Nissenbaum, 1996). Gender bias is a severe fairness issue in image search. Figure 1 shows an example: given a gender- neutral natural language query “a person is cook- ing”, only 2 out of 10 images retrieved by an image search model (Radford et al., 2021) depict females, while equalized exposure for male and female is expected. Such gender-biased search results are harmful to society as they change people’s cog- nition and worsen gender stereotypes (Kay et al., 2015). Mitigating gender bias in image search is imperative for social good. In this paper, we formally develop a framework for quantifying gender bias in image search results, where text queries in English1 are made gender- neutral, and gender-balanced search images are expected for models to retrieve. To evaluate model fairness, we use the normalized difference between masculine and feminine images in the retrieved re- sults to represent gender bias. We diagnose the gender bias of two primary families of multimodal models for image search: (1) the specialized mod- els that are often trained on in-domain datasets to perform text-image retrieval, and (2) the general- purpose representation models that are pre-trained on massive image and text data available online and can be applied to image search. Our analysis on MS-COCO (Lin et al., 2014) and Flickr30K (Young et al., 2014) datasets reveals that both types of mod- els lead to serious gender bias issues (e.g., nearly 70% …

Content truncated to 500 words