Longtonotes: OntoNotes with Longer Coreference Chains

Findings of the Association for Computational Linguistics: EACL 2023, pages 1428–1442 May 2-6, 2023 ©2023 Association for Computational Linguistics LongtoNotes: OntoNotes with Longer Coreference Chains Kumar Shridhar † Nicholas Monath ∗ ‡ Raghuveer Thirukovalluru♢ Alessandro Stolfo † Manzil Zaheer ¶ Andrew McCallum ‡ Mrinmaya Sachan † † ETH Zürich ‡ UMass Amherst ♢ Duke University ¶ Google Abstract Ontonotes has served as the most important benchmark for coreference resolution. How- ever, for ease of annotation, long documents in Ontonotes were split into smaller parts. In this work, we build a corpus of coreference- annotated documents of significantly longer length than what is currently available. We do so by providing an accurate, manually- curated, merging of annotations from docu- ments that were split into multiple parts in the original Ontonotes annotation process (Prad- han et al., 2013). The resulting corpus, which we call LongtoNotes contains documents in multiple genres of the English language with varying lengths, the longest of which are up to 8x the length of documents in Ontonotes, and 2x those in Litbank. We evaluate state- of-the-art neural coreference systems on this new corpus, analyze the relationships between model architectures/hyperparameters and doc- ument length on performance and efficiency of the models, and demonstrate areas of im- provement in long-document coreference mod- elling revealed by our new corpus. Our data and code is available at: https://github. com/kumar-shridhar/LongtoNotes. 1 Introduction Coreference resolution is an important problem in discourse with applications in knowledge-base con- struction (Luan et al., 2018), question-answering (Reddy et al., 2019) and reading assistants (Azab et al., 2013; Head et al., 2021). In many such set- tings, the documents of interest, are significantly longer and/or on wider varieties of domains than the currently available corpora with coreference annotation (Pradhan et al., 2013; Bamman et al., 2019; Mohan and Li, 2019; Cohen et al., 2017). The Ontonotes corpus (Pradhan et al., 2013) is perhaps the most widely used benchmark for coref- erence (Lee et al., 2013a; Durrett and Klein, 2013; ∗Now at Google. bn mz nw pt tc wb bc Genre 0 500 1000 1500 2000 2500 3000 3500 4000 Tokens/Document Dataset Longtonotes Ontonotes Figure 1: Comparing Average Document Length. Long documents in genres such as broadcast conversa- tions (bc) were split into smaller parts in Ontonotes. Our proposed dataset, LongtoNotes, restores documents to their original form, revealing dramatic increases in length in certain genres. Sachan et al., 2015; Wiseman et al., 2016; Lee et al., 2017; Joshi et al., 2020; Toshniwal et al., 2020b; Thirukovalluru et al., 2021; Kirstain et al., 2021). The construction process for Ontonotes, however, resulted in documents with an artificially reduced length. For ease of annotation, longer documents were split into smaller parts and each part was an- notated separately and treated as an independent document (Pradhan et al., 2013). The result is a corpus in which certain genres, such as broadcast conversation (bc), have greatly reduced length com- pared to their original form (Figure 1). As a result, the long, bursty spread of coreference …

Content truncated to 500 words