Combining Sample Selection and Error-Driven Pruning for Machine Learning of Coreference Rules

Combining Sample Selection and Error-Driven Pruning for Machine Learning of Coreference Rules Vincent Ng and Claire Cardie Department of Computer Science Cornell University Ithaca, NY 14853-7501 � yung,cardie � @cs.cornell.edu Abstract Most machine learning solutions to noun phrase coreference resolution recast the problem as a classification task. We ex- amine three potential problems with this reformulation, namely, skewed class dis- tributions, the inclusion of “hard” training instances, and the loss of transitivity in- herent in the original coreference relation. We show how these problems can be han- dled via intelligent sample selection and error-driven pruning of classification rule- sets. The resulting system achieves an F- measure of 69.5 and 63.4 on the MUC- 6 and MUC-7 coreference resolution data sets, respectively, surpassing the perfor- mance of the best MUC-6 and MUC-7 coreference systems. In particular, the system outperforms the best-performing learning-based coreference system to date. 1 Introduction Noun phrase coreference resolution refers to the problem of determining which noun phrases (NPs) refer to each real-world entity mentioned in a doc- ument. Machine learning approaches to this prob- lem have been reasonably successful, operating pri- marily by recasting the problem as a classification task (e.g. Aone and Bennett (1995), McCarthy and Lehnert (1995), Soon et al. (2001)). Specifically, an inductive learning algorithm is used to train a classi- fier that decides whether or not two NPs in a docu- ment are coreferent. Training data are typically cre- ated by relying on coreference chains from the train- ing documents: training instances are generated by pairing each NP with each of its preceding NPs; in- stances are labeled as positive if the two NPs are in the same coreference chain, and labeled as negative otherwise.1 A separate clustering mechanism then coordinates the possibly contradictory pairwise coreference clas- sification decisions and constructs a partition on the set of NPs with one cluster for each set of corefer- ent NPs. Although, in principle, any clustering algo- rithm can be used, most previous work uses a single- link clustering algorithm to impose coreference par- titions.2 An implicit assumption in the choice of the single-link clustering algorithm is that coreference resolution is viewed as anaphora resolution, i.e. the goal during clustering is to find an antecedent for each anaphoric NP in a document.3 Three intrinsic properties of coreference4, how- ever, make the formulation of the problem as a classification-based single-link clustering task po- tentially undesirable: Coreference is a rare relation. That is, most NP pairs in a document are not coreferent. Con- 1Two NPs are in the same coreference chain if and only if they are coreferent. 2One exception is Kehler’s work on probabilistic corefer- ence (Kehler, 1997), in which he applies Dempster’s Rule of Combination (Dempster, 1968) to combine all pairwise proba- bilities of coreference to form a partition. 3In this paper, we consider an NP anaphoric if it is part of a coreference chain but is not the head of the chain. 4Here, we use the term coreference loosely to refer to either the problem …

Content truncated to 500 words