The (Undesired) Attenuation of Human Biases by Multilinguality

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2056 - 2077 December 7-11, 2022 ©2022 Association for Computational Linguistics The (Undesired) Attenuation of Human Biases by Multilinguality Cristina España-Bonet DFKI GmbH, Saarland Informatics Campus Saarbrüken, Germany cristinae@dfki.de Alberto Barrón-Cedeño Universitá di Bologna Forlì, Italy a.barron@unibo.it Abstract Some human preferences are universal. The odor of vanilla is perceived as pleasant all around the world. We expect neural models trained on human texts to exhibit these kind of preferences, i.e. biases, but we show that this is not always the case. We explore 16 static and contextual embedding models in 9 languages and, when possible, compare them under sim- ilar training conditions. We introduce and re- lease CA-WEAT, multilingual cultural aware tests to quantify biases, and compare them to previous English-centric tests. Our experi- ments confirm that monolingual static embed- dings do exhibit human biases, but values dif- fer across languages, being far from universal. Biases are less evident in contextual models, to the point that the original human associa- tion might be reversed. Multilinguality proves to be another variable that attenuates and even reverses the effect of the bias, specially in contextual multilingual models. In order to explain this variance among models and lan- guages, we examine the effect of asymmetries in the training corpus, departures from isomor- phism in multilingual embedding spaces and discrepancies in the testing measures between languages. 1 Introduction The perception of odor pleasantness has been shown to be universal. Even if variations across individuals exist, they do not depend on culture (Ar- shamian et al., 2022). We call this is a human bias: judging a phenomenon in terms of values that are inherent in human beings, regardless of their socio- cultural background. On the contrary, a cultural bias is “the tendency to interpret and judge phenom- ena in terms of the distinctive values, beliefs, and other characteristics of the society or community to which one belongs”.1 Racism, sexism, ageism, 1https://dictionary.apa.org/ cultural-bias etc. are all social cultural biases and can be more or less present in distinct communities. As long as neural systems are trained on general- domain texts written by humans, one would expect and desire human biases to be present in embed- ding models for all languages. One would also expect (but not always desire) cultural biases, but these would depend on the language or the culture behind it. Lots of work has been done on detect- ing social cultural biases and trying to mitigate them (Bolukbasi et al., 2016; Zhao et al., 2018; Gonen and Goldberg, 2019; Ravfogel et al., 2020; Dev et al., 2020; Schick et al., 2021; Zhou et al., 2022). Also recent work has investigated the pres- ence of human biases in static word embeddings; first in English (Caliskan et al., 2017) and later in other languages (Lauscher and Glavaš, 2019; Lauscher et al., 2020a,b). These works make use of the Word Embedding Association Test (WEAT) for English —lists of concepts and attributes that hide implicit …

Content truncated to 500 words