How Conservative are Language Models? Adapting to the Introduction of Gender-Neutral Pronouns

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3624 - 3630 July 10-15, 2022 ©2022 Association for Computational Linguistics How Conservative are Language Models? Adapting to the Introduction of Gender-Neutral Pronouns Stephanie Brandl, Ruixiang Cui, Anders Søgaard University of Copenhagen, Denmark {brandl, rc, soegaard}@di.ku.dk Abstract Gender-neutral pronouns have recently been introduced in many languages to a) include non-binary people and b) as a generic singu- lar. Recent results from psycholinguistics sug- gest that gender-neutral pronouns (in Swedish) are not associated with human processing diffi- culties. This, we show, is in sharp contrast with automated processing. We show that gender-neutral pronouns in Danish, English, and Swedish are associated with higher per- plexity, more dispersed attention patterns, and worse downstream performance. We argue that such conservativity in language models may limit widespread adoption of gender-neutral pronouns and must therefore be resolved. 1 Introduction Many linguistic scholars have observed how tech- nology in general has altered the course of language evolution (Kristiansen et al., 2011; Abbasi, 2020), e.g., through the influence of social media conven- tions. Language technologies, in particular, have also been argued to have such effects, e.g., by re- ducing the pressure to acquire multiple languages. Gender-neutral pronouns is not an entirely mod- ern concept. In 1912, Ella Flag Young, then su- perintendent of the Chicago public-school system, said the following to a room full of school princi- pals: "The English language is in need of a personal pronoun of the third person, singular number, that will indicate both sexes and will thus eliminate our present awkwardness of speech." The use of gender-neutral pronouns has become much more popular in recent years (Gustafsson Sendén et al., 2021). In 2013, a gender-neutral pronoun was po- litically introduced in Swedish (Gustafsson Sendén et al., 2015) which can be used for both, people identifying outside the gender dichotomy and as a generic pronoun where information about gender is either unavailable or irrelevant. In a recently recorded eye-tracking study, Ver- goossen et al. (2020a) found no evidence that na- tive speakers of Swedish find it harder to pro- cess gender-neutral pronouns than gendered pro- nouns, an argument often brought up by oppo- nents of gender-inclusive language (Speyer and Schleef, 2019; Vergoossen et al., 2020b). In com- bination with their increasing popularity, this sug- gests gender-neutral pronouns have been or will be widely and fully adapted over time (Gustafs- son Sendén et al., 2015, 2021). However, since language technology has the potential to alter the course of language evolution, we want to make sure that our NLP models do not become a bottleneck for this positive development. Contribution We extract stimuli from a Swedish eye-tracking study that has shown no increase in processing cost in humans for the gender-neutral pronoun hen compared to gendered pronouns. We translate those stimuli into English and Danish and compare model perplexity across gendered and gender-neutral pronouns for all three languages. Furthermore, we systematically investigate perfor- mance differences across …

Content truncated to 500 words