Easy Victories and Uphill Battles in Coreference Resolution

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1971–1982, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Easy Victories and Uphill Battles in Coreference Resolution Greg Durrett and Dan Klein Computer Science Division University of California, Berkeley {gdurrett,klein}@cs.berkeley.edu Abstract Classical coreference systems encode various syntactic, discourse, and semantic phenomena explicitly, using heterogenous features com- puted from hand-crafted heuristics. In con- trast, we present a state-of-the-art coreference system that captures such phenomena implic- itly, with a small number of homogeneous feature templates examining shallow proper- ties of mentions. Surprisingly, our features are actually more effective than the corre- sponding hand-engineered ones at modeling these key linguistic phenomena, allowing us to win “easy victories” without crafted heuris- tics. These features are successful on syntax and discourse; however, they do not model semantic compatibility well, nor do we see gains from experiments with shallow seman- tic features from the literature, suggesting that this approach to semantics is an “uphill bat- tle.” Nonetheless, our final system1 outper- forms the Stanford system (Lee et al. (2011), the winner of the CoNLL 2011 shared task) by 3.5% absolute on the CoNLL metric and outperforms the IMS system (Bj¨orkelund and Farkas (2012), the best publicly available En- glish coreference system) by 1.9% absolute. 1 Introduction Coreference resolution is a multi-faceted task: hu- mans resolve references by exploiting contextual and grammatical clues, as well as semantic infor- mation and world knowledge, so capturing each of 1The Berkeley Coreference Resolution System is available at http://nlp.cs.berkeley.edu. these will be necessary for an automatic system to fully solve the problem. Acknowledging this com- plexity, coreference systems, either learning-based (Bengtson and Roth, 2008; Stoyanov et al., 2010; Haghighi and Klein, 2010; Rahman and Ng, 2011b) or rule-based (Haghighi and Klein, 2009; Lee et al., 2011), draw on diverse information sources and complex heuristics to resolve pronouns, model dis- course, determine anaphoricity, and identify seman- tically compatible mentions. However, this leads to systems with many heterogenous parts that can be difficult to interpret or modify. We build a learning-based, mention-synchronous coreference system that aims to use the simplest pos- sible set of features to tackle the various aspects of coreference resolution. Though they arise from a small number of simple templates, our features are numerous, which works to our advantage: we can both implicitly model important linguistic ef- fects and capture other patterns in the data that are not easily teased out by hand. As a result, our data- driven, homogeneous feature set is able to achieve high performance despite only using surface-level document characteristics and shallow syntactic in- formation. We win “easy victories” without design- ing features and heuristics explicitly targeting par- ticular phenomena. Though our approach is successful at modeling syntax, we find semantics to be a much more chal- lenging aspect of coreference. Our base system uses only two recall-oriented features on nominal and proper mentions: head match and exact string match. Building on these features, we critically eval- uate several classes …

Content truncated to 500 words