Word-Level Coreference Resolution

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7670–7675 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 7670 Word-Level Coreference Resolution Vladimir Dobrovolskii ABBYY / Moscow, Russia v.dobrovolskii@abbyy.com Abstract Recent coreference resolution models rely heavily on span representations to find coref- erence links between word spans. As the num- ber of spans is O(n2) in the length of text and the number of potential links is O(n4), vari- ous pruning techniques are necessary to make this approach computationally feasible. We propose instead to consider coreference links between individual words rather than word spans and then reconstruct the word spans. This reduces the complexity of the corefer- ence model to O(n2) and allows it to con- sider all potential mentions without pruning any of them out. We also demonstrate that, with these changes, SpanBERT for corefer- ence resolution will be significantly outper- formed by RoBERTa. While being highly effi- cient, our model performs competitively with recent coreference resolution systems on the OntoNotes benchmark. 1 Introduction Coreference resolution is used in various natu- ral language processing pipelines, such as ma- chine translation (Ohtani et al., 2019; Miculi- cich Werlen and Popescu-Belis, 2017), question answering (Dhingra et al., 2018) and text simplifi- cation (Wilkens et al., 2020). As a building block of larger systems itself, it is crucial that coreference resolution models aim to not only achieve higher benchmark scores but be time and memory efficient as well. Most recent coreference resolution models for English use the mention ranking approach: the highest ranking antecedent of each span is pre- dicted to be coreferent to that span. As this ap- proach presents a serious computational challenge of O(n4) complexity in the document length, Lee et al. (2017) keep only M top-scoring spans, while Lee et al. (2018) extend this idea by keeping only top K antecedents for each span based on an easily computable score. Their C2F-COREF model, which showed 72.6 F1 on the OntoNotes 5.0 shared task (Pradhan et al., 2012), was later improved by Joshi et al. (2020), who introduced SpanBERT to obtain better span representations (79.6 F1), and by Xu and Choi (2020), who proposed a novel way of higher-order coreference resolution (80.2 F1). The current state-of-the-art system built by Wu et al. (2020) uses a different approach, formulating the task as a machine reading comprehension prob- lem. While they were able to achieve an impressive 83.1 F1 on the OntoNotes 5.0 shared task, their model is particularly computationally expensive as it requires a full transformer pass to score each span’s antecedents. To make coreference resolution models more compact and allow for their easier incorporation into larger pipelines, we propose to separate the task of coreference resolution from span extrac- tion and to solve it on the word-level, lowering the complexity of the model to O(n2). The span ex- traction is to be performed separately only for those words that are found to be coreferent to some other words. An additional benefit of this approach is …

Content truncated to 500 words