Signed Coreference Resolution

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4950–4961 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4950 Signed Coreference Resolution Kayo Yin1, Kenneth DeHaan2, Malihe Alikhani3 1Language Technologies Institute, Carnegie Mellon University 2Department of American Sign Language, Gallaudet University 3Department of Computer Science, University of Pittsburgh kayoy@cs.cmu.edu, kenneth.de.haan@gallaudet.edu, malihe@pitt.edu Abstract Coreference resolution is key to many natural language processing tasks and yet has been relatively unexplored in Sign Language Pro- cessing. In signed languages, space is primar- ily used to establish reference. Solving coref- erence resolution for signed languages would not only enable higher-level Sign Language Processing systems, but also enhance our un- derstanding of language in different modalities and of situated references, which are key prob- lems in studying grounded language. In this paper, we: (1) introduce Signed Coreference Resolution (SCR), a new challenge for coref- erence modeling and Sign Language Process- ing; (2) collect an annotated corpus of German Sign Language with gold labels for corefer- ence together with an annotation software for the task; (3) explore features of hand gesture, iconicity, and spatial situated properties and move forward to propose a set of linguistically informed heuristics and unsupervised models for the task; (4) put forward several proposals about ways to address the complexities of this challenge effectively1. 1 Introduction While signed languages are fully-fledged natural languages with sophisticated grammatical systems that are fully comparable to those of spoken lan- guages (Emmorey, 2001), they are also in a com- pletely different modality with such extreme com- plexity that has yet to be thoroughly studied and understood. Much of our current language tech- nologies are not effective on signed languages, as natural language processing (NLP) modeling approaches are often based on linguistic theories of spoken languages, and expect either speech or written text as input. This results in technology that may be inaccessible to Deaf people where 1Our code, data and signed coreference annotation soft- ware are publicly available at https://github.com/ kayoyin/scr. signed languages are their primary mean of com- munication and who strongly prefer using their na- tive language than a spoken language (Padden and Humphries, 1988; Glickman and Hall, 2018), thus it is essential to extend NLP to signed languages. On the other hand, most of the recent research in Sign Language Processing (SLP) mainly focus on the visual component of signed languages and fail to address its linguistic challenges, such as corefer- ence resolution (Yin et al., 2021a). Coreference resolution is a critical component of natural language understanding and higher-level NLP applications including information extraction, text summarization, and machine translation, yet it is completely unexplored for signed languages. Although coreference relations have been studied in sign linguistics, computational models fall short in this area. Resolving coreference in signed lan- guages presents novel challenges as the meaning of pronominal signs are highly dependent on dis- course and spatial context (Cormier et al., 2010, 2013). Tackling this problem will help us gain a better understanding of how grounding is achieved across different …

Content truncated to 500 words