Unsupervised Models for Coreference Resolution

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 640–649, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Unsupervised Models for Coreference Resolution Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 vince@hlt.utdallas.edu Abstract We present a generative model for unsuper- vised coreference resolution that views coref- erence as an EM clustering process. For comparison purposes, we revisit Haghighi and Klein’s (2007) fully-generative Bayesian model for unsupervised coreference resolu- tion, discuss its potential weaknesses and con- sequently propose three modifications to their model. Experimental results on the ACE data sets show that our model outperforms their original model by a large margin and com- pares favorably to the modified model. 1 Introduction Coreference resolution is the problem of identifying which mentions (i.e., noun phrases) refer to which real-world entities. The availability of annotated coreference corpora produced as a result of the MUC conferences and the ACE evaluations has prompted the development of a variety of supervised machine learning approaches to coreference resolution in re- cent years. The focus of learning-based coreference research has also shifted from the acquisition of a pairwise model that determines whether two men- tions are co-referring (e.g., Soon et al. (2001), Ng and Cardie (2002), Yang et al. (2003)) to the de- velopment of rich linguistic features (e.g., Ji et al. (2005), Ponzetto and Strube (2006)) and the ex- ploitation of advanced techniques that involve joint learning (e.g., Daum´e III and Marcu (2005)) and joint inference (e.g., Denis and Baldridge (2007)) for coreference resolution and a related extraction task. The rich features, coupled with the increased complexity of coreference models, have made these supervised approaches more dependent on labeled data and less applicable to languages for which lit- tle or no annotated data exists. Given the growing importance of multi-lingual processing in the NLP community, however, the development of unsuper- vised and weakly supervised approaches for the au- tomatic processing of resource-scarce languages has become more important than ever. In fact, several popular weakly supervised learn- ing algorithms such as self-training, co-training (Blum and Mitchell, 1998), and EM (Dempster et al., 1977) have been applied to coreference resolu- tion (Ng and Cardie, 2003) and the related task of pronoun resolution (M¨uller et al., 2002; Kehler et al., 2004; Cherry and Bergsma, 2005). Given a small number of coreference-annotated documents and a large number of unlabeled documents, these weakly supervised learners aim to incrementally augment the labeled data by iteratively training a classifier1 on the labeled data and using it to label mention pairs randomly drawn from the unlabeled documents as COREFERENT or NOT COREFERENT. However, classifying mention pairs using such iterative ap- proaches is undesirable for coreference resolution: since the non-coreferent mention pairs significantly outnumber their coreferent counterparts, the result- ing classifiers generally have an increasing tendency to (mis)label a pair as non-coreferent as bootstrap- ping progresses (see Ng and Cardie (2003)). Motivated in part by these results, we present a generative, unsupervised model for probabilistically …

Content truncated to 500 words