Structured Generative Models for Unsupervised Named-Entity Clustering

Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, pages 164–172, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics Structured Generative Models for Unsupervised Named-Entity Clustering Micha Elsner, Eugene Charniak and Mark Johnson Brown Laboratory for Linguistic Information Processing (BLLIP) Brown University Providence, RI 02912 {melsner,ec,mj}@cs.brown.edu Abstract We describe a generative model for clustering named entities which also models named en- tity internal structure, clustering related words by role. The model is entirely unsupervised; it uses features from the named entity itself and its syntactic context, and coreference in- formation from an unsupervised pronoun re- solver. The model scores 86% on the MUC-7 named-entity dataset. To our knowledge, this is the best reported score for a fully unsuper- vised model, and the best score for a genera- tive model. 1 Introduction Named entity clustering is a classic task in NLP, and one for which both supervised and semi-supervised systems have excellent performance (Mikheev et al., 1998; Chinchor, 1998). In this paper, we describe a fully unsupervised system (using no “seed rules” or initial heuristics); to our knowledge this is the best such system reported on the MUC-7 dataset. In ad- dition, the model clusters the words which appear in named entities, discovering groups of words with similar roles such as first names and types of orga- nization. Finally, the model defines a notion of con- sistency between different references to the same en- tity; this component of the model yields a significant increase in performance. The main motivation for our system is the re- cent success of unsupervised generative models for coreference resolution. The model of Haghighi and Klein (2007) incorporated a latent variable for named entity class. They report a named entity score of 61.2 percent, well above the baseline of 46.4, but still far behind existing named-entity systems. We suspect that better models for named entities could aid in the coreference task. The easiest way to incorporate a better model is simply to run a super- vised or semi-supervised system as a preprocess. To perform joint inference, however, requires an unsu- pervised generative model for named entities. As far as we know, this work is the best such model. Named entities also pose another problem with the Haghighi and Klein (2007) coreference model; since it models only the heads of NPs, it will fail to resolve some references to named entities: (“Ford Motor Co.”, “Ford”), while erroneously merging others: (“Ford Motor Co.”, “Lockheed Martin Co.”). Ng (2008) showed that better features for match- ing named entities– exact string match and an “alias detector” looking for acronyms, abbreviations and name variants– improve the model’s performance substantially. Yet building an alias detector is non- trivial (Uryupina, 2004). English speakers know that “President Clinton” is the same person as “Bill Clin- ton” , not “President Bush”. But this cannot be im- plemented by simple substring matching. It requires some concept of the role of each word in the string. Our model attempts to learn …

Content truncated to 500 words