Multi-Field Information Extraction and Cross-Document Fusion

Proceedings of the 43rd Annual Meeting of the ACL, pages 483–490, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Multi-Field Information Extraction and Cross-Document Fusion Gideon S. Mann and David Yarowsky Department of Computer Science The Johns Hopkins University Baltimore, MD 21218 USA {gsm,yarowsky}@cs.jhu.edu Abstract In this paper, we examine the task of extracting a set of biographic facts about target individuals from a collection of Web pages. We automatically anno- tate training text with positive and negative exam- ples of fact extractions and train Rote, Na¨ıve Bayes, and Conditional Random Field extraction models for fact extraction from individual Web pages. We then propose and evaluate methods for fusing the extracted information across documents to return a consensus answer. A novel cross-field bootstrapping method leverages data interdependencies to yield improved performance. 1 Introduction Much recent statistical information extraction re- search has applied graphical models to extract in- formation from one particular document after train- ing on a large corpus of annotated data (Leek, 1997; Freitag and McCallum, 1999).1 Such systems are widely applicable, yet there remain many informa- tion extraction tasks that are not readily amenable to these methods. Annotated data required for training statistical extraction systems is sometimes unavail- able, while there are examples of the desired infor- mation. Further, the goal may be to find a few inter- related pieces of information that are stated multiple times in a set of documents. Here, we investigate one task that meets the above criteria. Given the name of a celebrity such as 1Alternatively, Riloff (1996) trains on in-domain and out-of-domain texts and then has a human filtering step. Huffman (1995) proposes a method to train a different type of extraction system by example. “Frank Zappa”, our goal is to extract a set of bio- graphic facts (e.g., birthdate, birth place and occupa- tion) about that person from documents on the Web. First, we describe a general method of automatic annotation for training from positive and negative examples and use the method to train Rote, Na¨ıve Bayes, and Conditional Random Field models (Sec- tion 2). We then examine how multiple extractions can be combined to form one consensus answer (Section 3). We compare fusion methods and show that frequency voting outperforms the single high- est confidence answer by an average of 11% across the various extractors. Increasing the number of re- trieved documents boosts the overall system accu- racy as additional documents which mention the in- dividual in question lead to higher recall. This im- proved recall more than compensates for a loss in per-extraction precision from these additional doc- uments. Next, we present a method for cross-field bootstrapping (Section 4) which improves per-field accuracy by 7%. We demonstrate that a small train- ing set with only the most relevant documents can be as effective as a larger training set with additional, less relevant documents (Section 5). 2 Training by Automatic Annotation Typically, statistical extraction systems (such as HMMs and CRFs) are trained using hand-annotated data. Annotating the necessary data …

Content truncated to 500 words