Conundrums in Entity Coreference Resolution: Making Sense of the State of the Art

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6620–6631, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6620 Conundrums in Entity Coreference Resolution: Making Sense of the State of the Art Jing Lu and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 {ljwinnie,vince}@hlt.utdallas.edu Abstract Despite the significant progress on entity coref- erence resolution observed in recent years, there is a general lack of understanding of what has been improved. We present an empir- ical analysis of state-of-the-art resolvers with the goal of providing the general NLP audi- ence with a better understanding of the state of the art and coreference researchers with di- rections for future research. 1 Introduction The advent of the neural NLP era has revolution- ized virtually all areas of NLP research. For entity coreference, many issues that were once thought to be important no longer appear to be particularly rel- evant to the current research agenda. Specifically, while a decade ago coreference researchers have focused on developing computational models that are complex (e.g., structured models (Fernandes et al., 2012; Bj¨orkelund and Kuhn, 2014; Martschat and Strube, 2015)) and knowledge-rich (e.g., those that encode world knowledge (Ponzetto and Strube, 2007; Rahman and Ng, 2011a; Hajishirzi et al., 2013)), nowadays virtually all state-of-the-art re- solvers employ a simple model (i.e., the mention- ranking model, which was developed more than a decade ago (Denis and Baldridge, 2008))1 and a fairly simple input representation (i.e., contex- tualized word embeddings) in conjunction with a mechanism for learning representations of entity mention spans such that coreferent mentions have similar representations (Lee et al., 2017, 2018; Kan- tor and Globerson, 2019; Joshi et al., 2019). Despite significant progress in the past few years in terms of performance numbers, what seems to be missing is an understanding of what has been 1The first learning-based resolver is a pairwise ranker (Con- nolly et al., 1994), which was extended by Denis and Baldridge (2008) to rank more than two candidate antecedents at a time. improved. The lack of understanding has long been a concern shared by coreference researchers, even before the neural revolution in NLP. This has led to several attempts to analyze coreference resolvers over the years (Stoyanov et al., 2009; Kummerfeld and Klein, 2013). With the development of neural resolvers, however, this concern has become more serious than ever: the fact that significant progress can be made via learning mention representations with a simple neural mention-ranking model that employs a fairly simple input representation for a task as challenging as coreference resolution (CR) is somewhat contrary to common wisdom. In light of this apparent conundrum, we present an empirical analysis of state-of-the-art entity coref- erence resolvers through four major sets of exper- iments in this paper, with the goal of gaining in- sights into their behaviors. We believe that our analysis will not only provide the general NLP au- dience with a better understanding of the state of the art, but also …

Content truncated to 500 words