Dual Cache for Long Document Neural Coreference Resolution

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 15272–15285 July 9-14, 2023 ©2023 Association for Computational Linguistics Dual Cache for Long Document Neural Coreference Resolution Qipeng Guo1, Xiangkun Hu1, Yue Zhang2, Xipeng Qiu3, Zheng Zhang1 1Amazon AWS AI, 2School of Engineering, Westlake University 3School of Computer Science, Fudan University {gqipeng, xiangkhu, zhaz}@amazon.com, zhangyue@westlake.edu.cn, xpqiu@fudan.edu.cn Abstract Recent works show the effectiveness of cache- based neural coreference resolution models on long documents. These models incrementally process a long document from left to right and extract relations between mentions and entities in a cache, resulting in much lower memory and computation cost compared to computing all mentions in parallel. However, they do not handle cache misses when high-quality entities are purged from the cache, which causes wrong assignments and leads to prediction errors. We propose a new hybrid cache that integrates two eviction policies to capture global and local entities separately, and effectively reduces the aggregated cache misses up to half as before, while improving F1 score of coreference by 0.7 ∼ 5.7pt. As such, the hybrid policy can accelerate existing cache-based models and of- fer a new long document coreference resolu- tion solution. Results show that our method outperforms existing methods on four bench- marks while saving up to 83% of inference time against non-cache-based models. Further, we achieve a new state-of-the-art on a long docu- ment coreference benchmark, LitBank. 1 Introduction Coreference Resolution (CR) is fundamental in document-level Natural Language Processing. Its goal is to identify mentions belonging to the same entity. These mentions often scatter throughout the document, but they can be linked together through coreference resolution. CR is a building block for common sense understanding (Levesque et al., 2012; Sakaguchi et al., 2021; Balahur et al., 2011; Liu et al., 2021a), reading comprehension (Dasigi et al., 2019; Storks et al., 2019), information extrac- tion (Yao et al., 2019; Ji and Grishman, 2008; Lu and Ng, 2018), and text summarization (Liu et al., 2021b; Azzam et al., 1999; Xu et al., 2020). Conventional CR models enumerate every pair of mentions in a document in parallel, so the com- Maria Skłodowska was born in Warsaw, in Congress Poland in the Russian Empire, on 7 November 1867 ... Paragraph 1 Born in Paris on 15 May 1859, Pierre Curie was the son of Eugène Curie (1827–1910), a doctor of French Catholic origin from Alsace, ... Paragraph 5 In nature, radium is found in uranium and (to a lesser extent) ... Paragraph 10 She won the Nobel Prize in Chemistry, "in recognition of her services to the advancement of chemistry by the discovery of the elements radium and polonium... Paragraph 15 Marie Curie Pierre Curie Radium Nobel Prize Figure 1: For a long document, the topic switching often happens, such as the discussion might shift from "Marie Curie" to "Pierre Curie," followed by "Radium," and then "Nobel Prize." These shifts in topic inevitably lead to certain entities being temporarily excluded from the ongoing discussion …

Content truncated to 500 words