Improving Coreference Resolution by Learning Entity-Level Distributed Representations

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 643–653, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Improving Coreference Resolution by Learning Entity-Level Distributed Representations Kevin Clark Computer Science Department Stanford University kevclark@cs.stanford.edu Christopher D. Manning Computer Science Department Stanford University manning@cs.stanford.edu Abstract A long-standing challenge in coreference resolution has been the incorporation of entity-level information – features defined over clusters of mentions instead of men- tion pairs. We present a neural net- work based coreference system that pro- duces high-dimensional vector represen- tations for pairs of coreference clusters. Using these representations, our system learns when combining clusters is de- sirable. We train the system with a learning-to-search algorithm that teaches it which local decisions (cluster merges) will lead to a high-scoring final corefer- ence partition. The system substantially outperforms the current state-of-the-art on the English and Chinese portions of the CoNLL 2012 Shared Task dataset despite using few hand-engineered features. 1 Introduction Coreference resolution, the task of identifying which mentions in a text refer to the same real- world entity, is fundamentally a clustering prob- lem. However, many recent state-of-the-art coref- erence systems operate solely by linking pairs of mentions together (Durrett and Klein, 2013; Martschat and Strube, 2015; Wiseman et al., 2015). An alternative approach is to use agglomera- tive clustering, treating each mention as a single- ton cluster at the outset and then repeatedly merg- ing clusters of mentions deemed to be referring to the same entity. Such systems can take advan- tage of entity-level information, i.e., features be- tween clusters of mentions instead of between just two mentions. As an example for why this is use- ful, it is clear that the clusters {Bill Clinton} and {Clinton, she} are not referring to the same entity, but it is ambiguous whether the pair of mentions Bill Clinton and Clinton are coreferent. Previous work has incorporated entity-level in- formation through features that capture hard con- straints like having gender or number agreement between clusters (Raghunathan et al., 2010; Dur- rett et al., 2013). In this work, we instead train a deep neural network to build distributed represen- tations of pairs of coreference clusters. This cap- tures entity-level information with a large number of learned, continuous features instead of a small number of hand-crafted categorical ones. Using the cluster-pair representations, our net- work learns when combining two coreference clusters is desirable. At test time it builds up coref- erence clusters incrementally, starting with each mention in its own cluster and then merging a pair of clusters each step. It makes these decisions with a novel easy-first cluster-ranking procedure that combines the strengths of cluster-ranking (Rah- man and Ng, 2011) and easy-first (Stoyanov and Eisner, 2012) coreference algorithms. Training incremental coreference systems is challenging because the coreference decisions fac- ing a model depend on previous decisions it has already made. We address this by using a learning-to-search algorithm inspired by SEARN (Daum´e III et al., 2009) to train our neural net- work. This …

Content truncated to 500 words