Unsupervised Coreference Resolution in a Nonparametric Bayesian Model

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 848–855, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Unsupervised Coreference Resolution in a Nonparametric Bayesian Model Aria Haghighi and Dan Klein Computer Science Division UC Berkeley {aria42, klein}@cs.berkeley.edu Abstract We present an unsupervised, nonparamet- ric Bayesian approach to coreference reso- lution which models both global entity iden- tity across a corpus as well as the sequen- tial anaphoric structure within each docu- ment. While most existing coreference work is driven by pairwise decisions, our model is fully generative, producing each mention from a combination of global entity proper- ties and local attentional state. Despite be- ing unsupervised, our system achieves a 70.3 MUC F1 measure on the MUC-6 test set, broadly in the range of some recent super- vised results. 1 Introduction Referring to an entity in natural language can broadly be decomposed into two processes. First, speakers directly introduce new entities into dis- course, entities which may be shared across dis- courses. This initial reference is typically accom- plished with proper or nominal expressions. Second, speakers refer back to entities already introduced. This anaphoric reference is canonically, though of course not always, accomplished with pronouns, and is governed by linguistic and cognitive constraints. In this paper, we present a nonparametric generative model of a document corpus which naturally con- nects these two processes. Most recent coreference resolution work has fo- cused on the task of deciding which mentions (noun phrases) in a document are coreferent. The domi- nant approach is to decompose the task into a col- lection of pairwise coreference decisions. One then applies discriminative learning methods to pairs of mentions, using features which encode properties such as distance, syntactic environment, and so on (Soon et al., 2001; Ng and Cardie, 2002). Although such approaches have been successful, they have several liabilities. First, rich features require plen- tiful labeled data, which we do not have for corefer- ence tasks in most domains and languages. Second, coreference is inherently a clustering or partitioning task. Naive pairwise methods can and do fail to pro- duce coherent partitions. One classic solution is to make greedy left-to-right linkage decisions. Recent work has addressed this issue in more global ways. McCallum and Wellner (2004) use graph partion- ing in order to reconcile pairwise scores into a final coherent clustering. Nonetheless, all these systems crucially rely on pairwise models because cluster- level models are much harder to work with, combi- natorially, in discriminative approaches. Another thread of coreference work has focused on the problem of identifying matches between documents (Milch et al., 2005; Bhattacharya and Getoor, 2006; Daume and Marcu, 2005). These methods ignore the sequential anaphoric structure inside documents, but construct models of how and when entities are shared between them.1 These models, as ours, are generative ones, since the fo- cus is on cluster discovery and the data is generally unlabeled. In this paper, we present a novel, fully genera- tive, nonparametric Bayesian model of mentions …

Content truncated to 500 words