Cross-Document Coreference on a Large Scale Corpus

Cross-Document Coreference on a Large Scale Corpus Abstract In this paper, we will compare and evaluate the effectiveness of different statistical methods in the task of cross-document coreference resolution. We created entity models for different test sets and compare the following disambiguation and clustering techniques to cluster the entity models in order to create coreference chains: Incremental Vector Space KL-Divergence Agglomerative Vector Space 1. Introduction Coreference analysis refers to the process of determining whether or not two mentions of entities refer to the same person (Kibble and Deemter, 2000). For example, consider the following short passage of text: John Smith was appointed chair of the committee. Because of his past experience, Mr. Smith seemed the perfect choice. His good friend John, however, was not considered. Coreference analysis attempts to decide whether John Smith and Mr. Smith refer to the same person, and whether John is also the same person. The task is often extended to include references such as his or even his good friend, though we do not explore that extension in this study. Addressing this problem is important to support systems such as those that search for, extract, and process mentions of “people of interest” in news or transcripts (BBN 2001), or for other information organization tasks that might benefit from precise knowledge of how names occur, such as Topic Detection and Tracking (Allan 2002). Cross-document coreference analysis pushes the task into considering whether mentions of a name in different documents are the same. The problem becomes more complex because documents might come from different sources, will probably have different authors and different writing conventions and styles(Bagga and Baldwin,1998), and may even be in different languages. There has been little published work on cross- document coreference analysis and that has generally been evaluated on a small corpus of documents. A major contribution of this work is to develop a substantially larger (more than two orders of magnitude) corpus for evaluation. We show that the previous approach is effective but that a variation on it, agglomerative vector space, provides improved and much more stable results. We begin in Section 2 by describing how cross- document coreference analysis is evaluated. We sketch prior work in Section 3 and describe our two evaluation corpora in Section 4. Section 5 discusses the three algorithms that we explore for this task and then Section 6 describes our experimental results on both corpora. In Section 7 we provide some additional analysis that attempts to explain some surprising results. We conclude in Section 8 with a description of our plans for future work. 2. Evaluation Given a collection of named entities from documents, the coreferencing task is to put them into equivalence classes, where every mention in the same class refers to the same entity (person, location, organization, and so on). The classes are referred to as “coreference chains” because the entities are chained together. To evaluate the coreference chains emitted by a system, we need truth data: the chains of entities that are actually …

Content truncated to 500 words