How Does Grammatical Gender Affect Noun Representations in Gender-Marking Languages?

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 463–471 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 463 How does Grammatical Gender Affect Noun Representations in Gender-Marking Languages? Hila Gonen1 Yova Kementchedjhieva2 Yoav Goldberg1,3 1Department of Computer Science, Bar-Ilan University 2University of Copenhagen 3Allen Institute for Artificial Intelligence hilagnn@gmail.com,yova@di.ku.dk,yoav.goldberg@gmail.com Abstract Many natural languages assign grammatical gender also to inanimate nouns in the lan- guage. In such languages, words that relate to the gender-marked nouns are inflected to agree with the noun’s gender. We show that this affects the word representations of inani- mate nouns, resulting in nouns with the same gender being closer to each other than nouns with different gender. While “embedding de- biasing” methods fail to remove the effect, we demonstrate that a careful application of meth- ods that neutralize grammatical gender signals from the words’ context when training word embeddings is effective in removing it. Fix- ing the grammatical gender bias yields a posi- tive effect on the quality of the resulting word embeddings, both in monolingual and cross- lingual settings. We note that successfully re- moving gender signals, while achievable, is not trivial to do and that a language-specific morphological analyzer, together with careful usage of it, are essential for achieving good re- sults. 1 Introduction Work on distributional word embeddings focuses almost exclusively on English, or on cross-lingual and language-agnostic techniques. However, lan- guages are diverse and different languages ex- hibit different linguistic phenomena, which may interact with the English-centric embedding learn- ing algorithms. In this work we look into one such phenomenon—grammatical gender—and ex- amine its effect on the learned representation. Many languages have rich grammatical sys- tems, that often include a complex gender system as well (Corbett, 1991). Languages with grammat- ical gender assign and morphologically mark gen- der not only to animate nouns (which have biologi- cal sex, e.g. man, woman, mother, father), but also to inanimate nouns (e.g. dream, book). This gram- matical gender assignment is mostly arbitrary: the same inanimate concept can have different gender in different languages. For example, a flower is masculine in Italian (fiore) and feminine in Ger- man (Blume). Languages often maintain an agreement system in which certain words agree on different morpho- logical features with other words they relate to. For example, English present-tense verbs are in- flected to agree with their nominal subject on the number feature. In other languages the agreement system is more elaborate, and in particular verbs, adjectives, determiners and other functions agree with nouns on many features, including gender (Corbett, 2006).1 Such grammatical agreement affects the distri- butional environment of nouns, as nouns of differ- ent gender become surrounded by different word forms: feminine nouns co-occur more with the feminine forms of words, while masculine nouns with the masculine forms. For example, the Italian word viaggio (“journey”-masc) will co-occur with durato (“last”-masc) and lungo (“long”-masc), while the word gita (“trip”-fem) will co-occur with durata (“last”-fem) and lunga (“long”-fem). Such changes in the distributional environment …

Content truncated to 500 words