Revealing the Myth of Higher-Order Inference in Coreference Resolution

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8527–8533, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8527 Revealing the Myth of Higher-Order Inference in Coreference Resolution Liyan Xu Computer Science Emory University, Atlanta, GA liyan.xu@emory.edu Jinho D. Choi Computer Science Emory University, Atlanta, GA jinho.choi@emory.edu Abstract This paper analyzes the impact of higher-order inference (HOI) on the task of coreference res- olution. HOI has been adapted by almost all recent coreference resolution models without taking much investigation on its true effective- ness over representation learning. To make a comprehensive analysis, we implement an end- to-end coreference system as well as four HOI approaches, attended antecedent, entity equal- ization, span clustering, and cluster merging, where the latter two are our original methods. We find that given a high-performing encoder such as SpanBERT, the impact of HOI is neg- ative to marginal, providing a new perspective of HOI to this task. Our best model using clus- ter merging shows the Avg-F1 of 80.2 on the CoNLL 2012 shared task dataset in English. 1 Introduction Coreference resolution has always been considered one of the unsolved NLP tasks due to its challeng- ing aspect of document-level understanding (Wise- man et al., 2015, 2016; Clark and Manning, 2015, 2016; Lee et al., 2017). Nonetheless, it has made a tremendous progress in recent years by adapting contextualized embedding encoders such as ELMo (Lee et al., 2018; Fei et al., 2019) and BERT (Kan- tor and Globerson, 2019; Joshi et al., 2019, 2020). The latest state-of-the-art model shows the improve- ment of 12.4% over the model introduced 2.5 years ago, where the major portion of the improvement is derived by representation learning (Figure 1). Most of these previous models have also adapted higher-order inference (HOI) for the global opti- mization of coreference links, although HOI clearly has not been the focus of those works, for the fact that gains from HOI have been reported marginal. This has inspired us to analyze the impact of HOI on modern coreference resolution models in order to envision the future direction of this research. To make thorough ablation studies among different approaches, we implement an end-to-end corefer- ence system in PyTorch (Sec 3.1), and two HOI approaches proposed by previous work, attended antecedent and entity equalization (Sec 3.2), along with two of our original approaches, span clustering and cluster merging (Sec 3.3). These approaches are experimented with two Transformer encoders, BERT and SpanBERT, to assess how effective HOI is even when coupled with those high-performing encoders (Sec 4). To the best of our knowledge, this is the first work to make a comprehensive anal- ysis on multiple HOI approaches side-by-side for the task of coreference resolution.1 62 65 68 71 74 77 80 W-16 C-16 L-17 L-18 F-19 K-19 J-19 J-20 Base ELMo BERT Higher-Order Table 1 Base ELMo BERT Higher-Order Wiseman et al. (2016) 63.39 0.82 Clark and Manning (2016) 64.76 0 0 0.53 Lee et al. (2017) 67.2 0 0 0 Lee et …

Content truncated to 500 words