Noun Phrase Coreference as Clustering

Noun Phrase Coreference as Clustering Claire Cardie and Kiri Wagstaff Department of Computer Science Cornell University Ithaca, NY 14853 E-mail: cardie,wkiri@cs.cornell.edu Abstract This paper introduces a new, unsupervised algo- rithm for noun phrase coreference resolution. It dif- fers from existing methods in that it views corer- erence resolution as a clustering task. In an eval- uation on the MUC-6 coreference resolution cor- pus, the algorithm achieves an F-measure of 53.6%~ placing it firmly between the worst (40%) and best (65%) systems in the MUC-6 evaluation. More im- portantly, the clustering approach outperforms the only MUC-6 system to treat coreference resolution as a learning problem. The clustering algorithm ap- pears to provide a flexible mechanism for coordi- nating the application of context-independent and context-dependent constraints and preferences for accurate partitioning of noun phrases into corefer- ence equivalence classes. 1 Introduction Many natural language processing (NLP) applica- tions require accurate noun phrase coreference reso- lution: They require a means for determining which noun phrases in a text or dialogue refer to the same real-world entity. The vast majority of algorithms for noun phrase coreference combine syntactic and, less often, semantic cues via a set of hand-crafted heuristics and filters. All but one system in the MUC-6 coreference performance evaluation (MUC, 1995), for example, handled coreference resolution in this manner. This same reliance on complicated hand-crafted algorithms is true even for the narrower task of pronoun resolution. Some exceptions exist, however. Ge et al. (1998) present a probabilistic model for pronoun resolution trained on a small sub- set of the Penn Treebank Wall Street Journal corpus (Marcus et al., 1993). Dagan and Itai (1991) develop a statistical filter for resolution of the pronoun "it" that selects among syntactically viable antecedents based on relevant subject-verb-object cooccurrences. Aone and Bennett (1995) and McCarthy and Lehn- ert (1995) employ decision tree algorithms to handle a broader subset of general noun phrase coreference problems. This paper presents a new corpus-based approach to noun phrase coreference. We believe that it is the first such unsupervised technique developed for the general noun phrase coreference task. In short, we view the task of noun phrase coreference resolution as a clustering task. First, each noun phrase in a document is represented as a vector of attribute-value pairs. Given the feature vector for each noun phrase, the clustering algorithm coor- dinates the application of context-independent and context-dependent coreference constraints and pref- erences to partition the noun phrases into equiv- alence classes, one class for each real-world entity mentioned in the text. Context-independent corefer- ence constraints and preferences are those that apply to two noun phrases in isolation. Context-dependent coreference decisions, on the other hand, consider the relationship of each noun phrase to surrounding noun phrases. In an evaluation on the MUC-6 coreference reso- lution corpus, our clustering approach achieves an F-measure of 53.6%, placing it firmly between the worst (40%) and best (65%) systems in the MUC- 6 evaluation. More importantly, the clustering ap- proach outperforms the only MUC-6 system to view …

Content truncated to 500 words