A Joint Framework for Coreference Resolution and Mention Head Detection

Proceedings of the 19th Conference on Computational Language Learning, pages 12–21, Beijing, China, July 30-31, 2015. c⃝2015 Association for Computational Linguistics A Joint Framework for Coreference Resolution and Mention Head Detection Haoruo Peng Kai-Wei Chang Dan Roth University of Illinois, Urbana-Champaign Urbana, IL, 61801 {hpeng7,kchang10,danr}@illinois.edu Abstract In coreference resolution, a fair amount of research treats mention detection as a pre- processed step and focuses on developing algorithms for clustering coreferred men- tions. However, there are significant gaps between the performance on gold mentions and the performance on the real problem, when mentions are predicted from raw text via an imperfect Mention Detection (MD) module. Motivated by the goal of reduc- ing such gaps, we develop an ILP-based joint coreference resolution and mention head formulation that is shown to yield sig- nificant improvements on coreference from raw text, outperforming existing state-of- art systems on both the ACE-2004 and the CoNLL-2012 datasets. At the same time, our joint approach is shown to improve men- tion detection by close to 15% F1. One key insight underlying our approach is that identifying and co-referring mention heads is not only sufficient but is more robust than working with complete mentions. 1 Introduction Mention detection is rarely studied as a stand-alone research problem (Recasens et al. (2013) is one key exception). Most coreference resolution work simply mentions it in passing as a module in the pipelined system (Chang et al., 2013; Durrett and Klein, 2013; Lee et al., 2011; Bj¨orkelund and Kuhn, 2014). However, the lack of emphasis is not due to this being a minor issue, but rather, we think, its dif- ficulty. Indeed, many papers report results in terms of gold mentions versus system generated mentions, as shown in Table 1. Current state-of-the-art sys- tems show a very significant drop in performance when running on system generated mentions. These performance gaps are worrisome, since the real goal of NLP systems is to process raw data. System Dataset Gold Predict Gap Illinois CoNLL-12 77.05 60.00 17.05 Illinois CoNLL-11 77.22 60.18 17.04 Illinois ACE-04 79.42 68.27 11.15 Berkeley CoNLL-11 76.68 60.42 16.26 Stanford ACE-04 81.05 70.33 10.72 Table 1: Performance gaps between using gold mentions and predicted mentions for three state-of-the-art corefer- ence resolution systems. Performance gaps are always larger than 10%. Illinois’s system (Chang et al., 2013) is evaluated on CoNLL (2012, 2011) Shared Task and ACE-2004 datasets. It reports an average F1 score of MUC, B3 and CEAFe met- rics using CoNLL v7.0 scorer. Berkeley’s system (Durrett and Klein, 2013) reports the same average score on the CoNLL- 2011 Shared Task dataset. Results of Stanford’s system (Lee et al., 2011) are for B3 metric on ACE-2004 dataset. This paper focuses on improving end-to-end coreference performance. We do this by: 1) De- veloping a new ILP-based joint learning and infer- ence formulation for coreference and mention head detection. 2) Developing a better mention head can- didate generation algorithm. Importantly, we focus on heads rather than mention boundaries since those can be identified more robustly and …

Content truncated to 500 words