Entity-Based Cross-Document Coreferencing Using the Vector Space Model

Entity-Based Cross-Document Coreferencing Using the Vector Space Model Amit Bagga Box 90129 Dept. of Computer Science Duke University Durham, NC 27708-0129 amit@cs.duke.edu Breck Baldwin Institute for Research in Cognitive Sciences University of Pennsylvania 3401 Walnut St. 400C Philadelphia, PA 19104 breck@unagi.cis.upenn.edu Abstract Cross-document coreference occurs when the same person, place, event, or concept is discussed in more than one text source. Computer recognition of this phenomenon is important because it helps break "the document boundary" by allowing a user to ex- amine information about a particular entity from multiple text sources at the same time. In this paper we describe a cross-document coreference resolution algorithm which uses the Vector Space Model to re- solve ambiguities between people having the same name. In addition, we also describe a scoring algo- rithm for evaluating the cross-document coreference chains produced by our system and we compare our algorithm to the scoring algorithm used in the MUC- 6 (within document) coreference task. 1 Introduction Cross-document coreference occurs when the same person, place, event, or concept is discussed in more than one text source. Computer recognition of this phenomenon is important because it helps break "the document boundary" by allowing a user to ex- amine information about a particular entity from multiple text sources at the same time. In partic- ular, resolving cross-document coreferences allows a user to identify trends and dependencies across documents. Cross-document coreference can also be used as the central tool for producing summaries from multiple documents, and for information fu- sion, both of which have been identified as advanced areas of research by the TIPSTER Phase III pro- gram. Cross-document coreference was also identi- fied as one of the potential tasks for the Sixth Mes- sage Understanding Conference (MUC-6) but was not included as a formal task because it was consid- ered too ambitious (Grishman 94). In this paper we describe a highly successful cross- document coreference resolution algorithm which uses the Vector Space Model to resolve ambiguities between people having the same name. In addition, we also describe a scoring algorithm for evaluating the cross-document coreference chains produced by our system and we compare our algorithm to the scoring algorithm used in the MUC-6 (within docu- ment) coreference task. 2 Cross-Document Coreference: The Problem Cross-document corefereuce is a distinct technology from Named Entity recognizers like IsoQuest's Ne- tOwl and IBM's Textract because it attempts to determine whether name; matches are actually the same individual (not all John Smiths are the same). Neither NetOwl or Textract have mechanisms which try to keep same-named individuals distinct if they are different people. Cross-document coreference also differs in sub- stantial ways from within-document coreference. Within a document there is a certain amount of consistency which cannot be expected across doc- uments. In addition, the problems encountered dur- ing within document coreference are compounded when looking for coreferences across documents be- cause the underlying principles of linguistics and discourse context no longer apply across docu- ments. Because the underlying assumptions in cross- document coreference are …

Content truncated to 500 words