Debiasing Word Embeddings with Nonlinear Geometry

Proceedings of the 29th International Conference on Computational Linguistics, pages 1286–1298 October 12–17, 2022. 1286 Debiasing Word Embeddings with Nonlinear Geometry Lu Cheng Department of Computer Science, University of Illinois Chicago lucheng@uic.edu Nayoung Kim and Huan Liu School of Computing and Augmented Intelligence Arizona State University {nkim48,huanliu}@asu.edu Abstract Debiasing word embeddings has been largely limited to individual and independent social categories. However, real-world corpora typ- ically present multiple social categories that possibly correlate or intersect with each other. For instance, “hair weaves” is stereotypically associated with African American females, but neither African American nor females alone. Therefore, this work studies biases associated with multiple social categories: joint biases induced by the union of different categories and intersectional biases that do not overlap with the biases of the constituent categories. We first empirically observe that individual bi- ases intersect non-trivially (i.e., over a one- dimensional subspace). Drawing from the in- tersectional theory in social science and the lin- guistic theory, we then construct an intersec- tional subspace to debias for multiple social categories using the nonlinear geometry of in- dividual biases. Empirical evaluations corrob- orate the efficacy of our approach1. 1 Introduction Due to the reliance on the large-scale text corpora for training, it has been observed that word em- beddings are prone to express social biases inher- ent in the data (Bolukbasi et al., 2016; Caliskan et al., 2017). Prior research (e.g. Zhao et al., 2019; Bolukbasi et al., 2016) in debiasing word embed- dings mitigates biases associated with individual social categories and treats each category in iso- lation. For example, the seminal Hard-Debiasing approach (Bolukbasi et al., 2016) identifies the bias direction of a category (e.g., gender) and then re- moves the direction from the target word such that it is equidistant to all groups (e.g., female and male at a binary level with gender) in the category. How- ever, real-world training corpora typically present at1Data and implementation code can be downloaded https://github.com/GitHubLuCheng/ Implementation-of-JoSEC-COLING-22. Gender Race Gender Race Joint Biases: Gender ∪ Race Intersectional Bias: Gender ∩ Race Figure 1: Illustrations of joint (i.e., union) and intersec- tional (i.e., intersect) biases using Gender and Race. multiple social categories (e.g., gender and race), possibly with higher cardinality. These social cate- gories can further correlate or intersect with each other (Thomas, 2004; Hancock, 2007). Despite the promising results, debiasing for individual social categories limits our understanding of the complex nature of social biases. Alternatively, we might consider biases in the presence of multiple social categories (Foulds et al., 2020; Kearns et al., 2018; Cheng et al., 2022b,a). This can result in at least two scenarios, as depicted in Fig. 1. First, word embeddings can simulta- neously present multiple biases that might non- trivially correlate with each other. For example, debiasing for gender can influence the results of racial bias. In the literature of social psychology, a number of works (e.g. Akrami et al., 2011; Bierly, 1985; Allport et al., 1954) studied the interrelation- ship between various biases, a.k.a. “generalized prejudice”. We …

Content truncated to 500 words