Contrastive Representation Learning for Cross-Document Coreference Resolution of Events and Entities

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3644 - 3655 July 10-15, 2022 ©2022 Association for Computational Linguistics Contrastive Representation Learning for Cross-Document Coreference Resolution of Events and Entities Benjamin Hsu and Graham Horwood AWS AI Labs {benhsu, ghorwood}@amazon.com Abstract Identifying related entities and events within and across documents is fundamental to natu- ral language understanding. We present an ap- proach to entity and event coreference resolu- tion utilizing contrastive representation learn- ing. Earlier state-of-the-art methods have for- mulated this problem as a binary classification problem and leveraged large transformers in a cross-encoder architecture to achieve their re- sults. For large collections of documents and corresponding set of n mentions, the necessity of performing n2 transformer computations in these earlier approaches can be computation- ally intensive. We show that it is possible to re- duce this burden by applying contrastive learn- ing techniques that only require n transformer computations at inference time. Our method achieves state-of-the-art results on a number of key metrics on the ECB+ corpus and is com- petitive on others. 1 Introduction Coreference resolution is the fundamental NLP task of finding all mentions that refer to the same real world entity or event in text. It is an important step for higher level NLP tasks involving natural lan- guage understanding, such as text summarization (Azzam et al., 1999), information extraction (Ze- lenko et al., 2004), and question-answering (Vicedo and Ferrández, 2000). Historically, coreference resolution of entities in the same text document – within document (WD) coreference resolution – has received the most attention, though more re- cently focus has moved toward cross-document (CD) coreference resolution. CD coreference resolution has recently gained re- newed interest for its application in multi-document analysis tasks. CD coreference resolution presents unique challenges not found in the WD context. Spans of text come from different documents with- out any inherent linear order, and there is no notion that antecedents for a given expression typically oc- cur before the expression, as in a single document. Coreferent expressions also cannot be assumed to occur near one another. Furthermore, documents are also assumed to be authored independently and about different—though lexically similar—topics. For instance, the event described in the sentences from Topic 19 in Table 1 below are not coreferen- tial, despite their lexical similarity ("killed"). Another important aspect of CD coreference res- olution is the potential scale of the problem. In certain applications, the number of documents can be large and ever growing. In particular, for ap- plications that merge information from across doc- uments, such as multi-document summarization (Falke et al., 2017) or multi-hop question answer- ing (Dhingra et al., 2018), the corpus in question can be both large and dynamically increasing in size. Past methods of CD coreference resolution have treated the problem as a binary classification task: given two pairs of mentions, classify them as refer- ring to the same entity or not (Bejan and Harabagiu, 2010; Yang et …

Content truncated to 500 words