Examining Gender Bias in Languages with Grammatical Gender

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5276–5284, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5276 Examining Gender Bias in Languages with Grammatical Gender Pei Zhou1,2, Weijia Shi1, Jieyu Zhao1, Kuan-Hao Huang1, Muhao Chen1,3, Ryan Cotterell4, Kai-Wei Chang1 1Department of Computer Science, University of California Los Angeles 2Department of Computer Science, University of Southern California 3Department of Computer and Information Science, University of Pennsylvania 4Department of Computer Science, Johns Hopkins University peiz@usc.edu; {swj0419,jyzhao, khhuang, kwchang}@cs.ucla.edu; muhao@seas.upenn.edu; ryan.cotterell@jhu.edu Abstract Recent studies have shown that word embed- dings exhibit gender bias inherited from the training corpora. However, most studies to date have focused on quantifying and mitigat- ing such bias only in English. These analy- ses cannot be directly extended to languages that exhibit morphological agreement on gen- der, such as Spanish and French. In this paper, we propose new metrics for evaluating gender bias in word embeddings of these languages and further demonstrate evidence of gender bias in bilingual embeddings which align these languages with English. Finally, we extend an existing approach to mitigate gender bias in word embedding under both monolingual and bilingual settings. Experiments on modified Word Embedding Association Test, word sim- ilarity, word translation, and word pair transla- tion tasks show that the proposed approaches effectively reduce the gender bias while pre- serving the utility of the embeddings. 1 Introduction Word embeddings are widely used in modern nat- ural language processing tools. By virtue of their being trained on large, human-written corpora, re- cent studies have shown that word embeddings, in addition to capturing a word’s semantics, also en- code gender bias in society (Bolukbasi et al., 2016; Caliskan et al., 2017). As a result of this bias, the embeddings may cause undesired consequences in the resulting models (Zhao et al., 2018a; Font and Costa-juss`a, 2019). Therefore, extensive ef- fort has been put toward analyzing and mitigating gender bias in word embeddings (Bolukbasi et al., 2016; Zhao et al., 2018b; Dev and Phillips, 2019; Ethayarajh et al., 2019). Existing studies on gender bias almost exclu- sively focus on English (EN) word embeddings. Unfortunately, the techniques used to mitigate bias in English word embeddings cannot be directly applied to languages with grammatical gender1, where all nouns are assigned a gender class and the corresponding dependent articles, adjectives, and verbs must agree in gender with the noun (e.g. in Spanish: la buena enfermera the good female nurse, el buen enfermero the good male nurse) (Cor- bett, 1991, 2006). This is because most existing ap- proaches define bias in word embeddings based on the projection of a word on a gender direction (e.g. “nurse” in English is biased because its projection on the gender direction inclines towards female). When the grammatical gender exists, such bias def- inition is problematic as masculine and feminine words naturally carry gender information. For ex- ample, “enfermero” (male nurse) and “enfermera” (female nurse) lean toward male …

Content truncated to 500 words