Decentralized Entity-Level Modeling for Coreference Resolution

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 114–124, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Decentralized Entity-Level Modeling for Coreference Resolution Greg Durrett, David Hall, and Dan Klein Computer Science Division University of California, Berkeley {gdurrett,dlwh,klein}@cs.berkeley.edu Abstract Efficiently incorporating entity-level in- formation is a challenge for coreference resolution systems due to the difficulty of exact inference over partitions. We de- scribe an end-to-end discriminative prob- abilistic model for coreference that, along with standard pairwise features, enforces structural agreement constraints between specified properties of coreferent men- tions. This model can be represented as a factor graph for each document that ad- mits efficient inference via belief propaga- tion. We show that our method can use entity-level information to outperform a basic pairwise system. 1 Introduction The inclusion of entity-level features has been a driving force behind the development of many coreference resolution systems (Luo et al., 2004; Rahman and Ng, 2009; Haghighi and Klein, 2010; Lee et al., 2011). There is no polynomial-time dy- namic program for inference in a model with ar- bitrary entity-level features, so systems that use such features typically rely on making decisions in a pipelined manner and sticking with them, op- erating greedily in a left-to-right fashion (Rahman and Ng, 2009) or in a multi-pass, sieve-like man- ner (Raghunathan et al., 2010). However, such systems may be locked into bad coreference deci- sions and are difficult to directly optimize for stan- dard evaluation metrics. In this work, we present a new structured model of entity-level information designed to allow effi- cient inference. We use a log-linear model that can be expressed as a factor graph. Pairwise features appear in the model as unary factors, adjacent to nodes representing a choice of antecedent (or none) for each mention. Additional nodes model entity-level properties on a per-mention basis, and structural agreement factors softly drive properties of coreferent mentions to agree with one another. This is a key feature of our model: mentions man- age their partial membership in various corefer- ence chains, so that information about entity-level properties is decentralized and propagated across individual mentions, and we never need to explic- itly instantiate entities. Exact inference in this factor graph is in- tractable, but efficient approximate inference can be carried out with belief propagation. Our model is the first discriminatively-trained model that both makes joint decisions over an entire document and models specific entity-level properties, rather than simply enforcing transitivity of pairwise decisions (Finkel and Manning, 2008; Song et al., 2012). We evaluate our system on the dataset from the CoNLL 2011 shared task using three differ- ent types of properties: synthetic oracle proper- ties, entity phi features (number, gender, animacy, and NER type), and properties derived from un- supervised clusters targeting semantic type infor- mation. In all cases, our transitive model of en- tity properties equals or outperforms our pairwise system and our reimplementation of a previous entity-level system (Rahman and Ng, 2009). Our final system is competitive …

Content truncated to 500 words