Using Automatically Extracted Minimum Spans to Disentangle Coreference Evaluation from Boundary Detection

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4168–4178 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4168 Using Automatically Extracted Minimum Spans to Disentangle Coreference Evaluation from Boundary Detection Nafise Sadat Moosavi1 and Leo Born2 and Massimo Poesio3 and Michael Strube4 1Ubiquitous Knowledge Processing (UKP) Lab, Technische Universit¨at Darmstadt 2Institute for Computational Linguistics, Heidelberg University 3School of Electronic Engineering and Computer Science, Queen Mary University of London 4Heidelberg Institute for Theoretical Studies gGmbH Abstract The common practice in coreference resolu- tion is to identify and evaluate the maximum span of mentions. The use of maximum spans tangles coreference evaluation with the challenges of mention boundary detection like prepositional phrase attachment. To address this problem, minimum spans are manually annotated in smaller corpora. However, this additional annotation is costly and therefore, this solution does not scale to large corpora. In this paper, we propose the MINA algo- rithm for automatically extracting minimum spans to benefit from minimum span evalu- ation in all corpora. We show that the ex- tracted minimum spans by MINA are consis- tent with those that are manually annotated by experts. Our experiments show that us- ing minimum spans is in particular impor- tant in cross-dataset coreference evaluation, in which detected mention boundaries are noisier due to domain shift. We have inte- grated MINA into https://github.com/ ns-moosavi/coval for reporting standard coreference scores based on both maximum and automatically detected minimum spans. 1 Introduction Coreference resolution is the task of finding differ- ent expressions that refer to the same real-world entity. Each referring expression is called a men- tion. The common approach to annotate corefer- ring mentions is to specify the largest span of each mention. The problem with using maximum spans in coreference evaluation is that a single mention may have different maximum boundaries based on gold vs. automatically detected syntactic struc- tures. For instance, variations in prepositional phrase attachment, which is a known challenge in syntactic parsing, will lead to different maximum boundaries for a single mention. In order to decouple coreference evaluation from maximum boundary detection complexities, smaller corpora like MUC (Hirschman and Chin- chor, 1997), ACE (Mitchell et al., 2002), and AR- RAU (Uryupina et al., 2016) explicitly annotate the minimum span as well as the maximum logi- cal span of each mention. The annotated minimum spans indicate the minimum strings that a corefer- ence resolver must identify for the corresponding mentions. This solution comes with an additional annotation cost. As a result, the annotation of min- imum spans has been discarded in larger corpora like CoNLL-2012 (Pradhan et al., 2012). In this paper, we propose MINA, a MINimum span extraction Algorithm that automatically de- termines minimum spans from constituency-based parse trees. Based on our analyses, MINA spans are compatible with those that are manually anno- tated by experts. By using MINA, we can bene- fit from minimum span evaluation for all corpora without introducing additional annotation costs. While the use of MINA spans …

Content truncated to 500 words