Journalistic Guidelines Aware News Image Captioning

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 518–532 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 518 Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models Robert Wolfe University of Washington rwolfe3@uw.edu Aylin Caliskan University of Washington aylin@uw.edu Abstract We use a dataset of U.S. first names with la- bels based on predominant gender and racial group to examine the effect of training corpus frequency on tokenization, contextualization, similarity to initial representation, and bias in BERT, GPT-2, T5, and XLNet. We show that predominantly female and non-white names are less frequent in the training corpora of these four language models. We find that infre- quent names are more self-similar across con- texts, with Spearman’s ρ between frequency and self-similarity as low as −.763. Infre- quent names are also less similar to initial rep- resentation, with Spearman’s ρ between fre- quency and linear centered kernel alignment (CKA) similarity to initial representation as high as .702. Moreover, we find Spearman’s ρ between racial bias and name frequency in BERT of .492, indicating that lower-frequency minority group names are more associated with unpleasantness. Representations of infre- quent names undergo more processing, but are more self-similar, indicating that models rely on less context-informed representations of un- common and minority names which are overfit to a lower number of observed contexts. 1 Introduction Human social perception is linked to frequency of observation. Hughes et al. (2019) show using func- tional magnetic resonance imaging (fMRI) scans that humans are more aware of variation in the faces of members of their own race, and perceive members of other races as repeated instances of a social class, rather than as individuals. Most people interact more with individuals of their own race, and develop better cognitive skills for differentiat- ing members of the race they see most frequently. Recent research indicates that state-of-the-art AI systems mirror such biased and unequal human perceptions. For example, Buolamwini and Gebru (2018) show that under-representation in the train- ing data of computer vision models causes poor performance on classification tasks for women and minority racial groups. First names are used in both social psychology and Natural Language Processing (NLP) as a proxy to ground truth data for studying racial and gender biases. The implicit association test (IAT) of Green- wald et al. (1998) finds that study participants per- ceive European-American names as more pleasant than African-American names, and Caliskan et al. (2017) demonstrate with the word embedding as- sociation test (WEAT) that human biases observed in the IAT exist in static word embeddings, learned vector representations of words widely used in NLP. We use a list of first names labeled by gender and racial group based on U.S. Social Security Admin- istration data and the names dataset of Tzioumis (2018) to analyze how name frequency affects mi- nority social groups in four neural language models: BERT, GPT-2, XLNet, and T5. Neural language models have advanced the state of the art in NLP, and are found …

Content truncated to 500 words