First-Order Probabilistic Models for Coreference Resolution

Proceedings of NAACL HLT 2007, pages 81–88, Rochester, NY, April 2007. c⃝2007 Association for Computational Linguistics First-Order Probabilistic Models for Coreference Resolution Aron Culotta and Michael Wick and Andrew McCallum Department of Computer Science University of Massachusetts Amherst, MA 01003 {culotta,mwick,mccallum}@cs.umass.edu Abstract Traditional noun phrase coreference res- olution systems represent features only of pairs of noun phrases. In this paper, we propose a machine learning method that enables features over sets of noun phrases, resulting in a first-order proba- bilistic model for coreference. We out- line a set of approximations that make this approach practical, and apply our method to the ACE coreference dataset, achiev- ing a 45% error reduction over a com- parable method that only considers fea- tures of pairs of noun phrases. This result demonstrates an example of how a first- order logic representation can be incorpo- rated into a probabilistic model and scaled efficiently. 1 Introduction Noun phrase coreference resolution is the problem of clustering noun phrases into anaphoric sets. A standard machine learning approach is to perform a set of independent binary classifications of the form “Is mention a coreferent with mention b?” This approach of decomposing the problem into pairwise decisions presents at least two related diffi- culties. First, it is not clear how best to convert the set of pairwise classifications into a disjoint cluster- ing of noun phrases. The problem stems from the transitivity constraints of coreference: If a and b are coreferent, and b and c are coreferent, then a and c must be coreferent. This problem has recently been addressed by a number of researchers. A simple approach is to per- form the transitive closure of the pairwise decisions. However, as shown in recent work (McCallum and Wellner, 2003; Singla and Domingos, 2005), bet- ter performance can be obtained by performing rela- tional inference to directly consider the dependence among a set of predictions. For example, McCal- lum and Wellner (2005) apply a graph partitioning algorithm on a weighted, undirected graph in which vertices are noun phrases and edges are weighted by the pairwise score between noun phrases. A second and less studied difficulty is that the pairwise decomposition restricts the feature set to evidence about pairs of noun phrases only. This re- striction can be detrimental if there exist features of sets of noun phrases that cannot be captured by a combination of pairwise features. As a simple exam- ple, consider prohibiting coreferent sets that consist only of pronouns. That is, we would like to require that there be at least one antecedent for a set of pro- nouns. The pairwise decomposition does not make it possible to capture this constraint. In general, we would like to construct arbitrary features over a cluster of noun phrases using the full expressivity of first-order logic. Enabling this sort of flexible representation within a statistical model has been the subject of a long line of research on first-order probabilistic models (Gaifman, 1964; Halpern, 1990; Paskin, 2002; Poole, 2003; Richard- son and Domingos, …

Content truncated to 500 words