Recognition of They/Them as Singular Personal Pronouns in Coreference Resolution

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3426 - 3432 July 10-15, 2022 ©2022 Association for Computational Linguistics Recognition of They/Them as Singular Personal Pronouns in Coreference Resolution Connor Baumler University of Maryland, College Park baumler@umd.edu Rachel Rudinger University of Maryland, College Park rudinger@umd.edu Abstract As using they/them as personal pronouns be- comes increasingly common in English, it is important that coreference resolution systems work as well for individuals who use personal “they” as they do for those who use gendered personal pronouns. We introduce a new bench- mark for coreference resolution systems which evaluates singular personal “they” recognition. Using these WinoNB schemas, we evaluate a number of publicly available coreference res- olution systems and confirm their bias toward resolving “they” pronouns as plural. 1 Introduction While singular “they” has been widely used in the English-speaking world as a personal pro- noun among nonbinary and other members of the LGBTQIA+ community for many years, it has taken time for this usage to be accepted by cis- normative society at large. In particular, profes- sional institutions’ acceptance of singular personal “they” has been much slower, but seen some up- take in recent years. The American Psychological Association began accepting singular “they” as a “self-identified pronoun” in 2019,1 the same year Merriam-Webster added “nonbinary they” to their dictionary and made it their word of the year. 2 As a result, English language models trained exclusively on sources such as old newspaper arti- cles are likely not exposed to examples of singular “they” usage, in either the personal or generic set- ting. Even models trained on more recent data may miss this usage entirely if they rely on text that was subjected to conservative style guides. This means even state-of-the-art coreference resolution systems may not have recognition of singular “they.” In this work, we examine whether existing coref- erence resolutions systems are able to correctly 1Via APA Style 2Via Merriam-Webster (1a) The paramedic performed CPR on the pas- senger even though they knew it was too late. (1b) The paramedic performed CPR on the pas- senger even though they were already dead. (2a) The paramedics tried to help Riley even though they knew it was too late. (2b) The paramedics tried to help Riley even though they were already dead. Figure 1: A (1) “Winogender” and corresponding (2) “WinoNB schema”. The correct answers are bolded. resolve cases of singular personal “they”. We find that when given a choice between resolving “they” correctly to a singular, named entity or to a group, current systems overwhelmingly choose to resolve “they” as plural or even choose not to resolve the pronoun at all. We also investigate these systems’ recognition of singular generic “they” and find that this is a much more easily recognized use-case for some models. Overall, we find that models which have been trained on more contemporary or stylis- tically varied text that may contain examples of singular “they” to have …

Content truncated to 500 words