Redundancy-based Correction of Automatically Extracted Facts

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 57–64, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Redundancy-based Correction of Automatically Extracted Facts Roman Yangarber and Lauri Jokipii Department of Computer Science University of Helsinki, Finland first.last@cs.helsinki.fi Abstract The accuracy of event extraction is lim- ited by a number of complicating factors, with errors compounded at all sages in- side the Information Extraction pipeline. In this paper, we present methods for re- covering automatically from errors com- mitted in the pipeline processing. Recov- ery is achieved via post-processing facts aggregated over a large collection of doc- uments, and suggesting corrections based on evidence external to the document. A further improvement is derived from prop- agating multiple, locally non-best slot fills through the pipeline. Evaluation shows that the global analysis is over 10 times more likely to suggest valid corrections to the local-only analysis than it is to suggest erroneous ones. This yields a substantial overall gain, with no supervised training. 1 Introduction Information Extraction (IE) is a technology for find- ing facts in plain text, and coding them in a logical representation, such as, e.g., a relational database. IE is typically viewed and implemented as a se- quence of stages—a “pipeline”: 1. Layout, tokenization, lexical analysis 2. Name recognition and classification 3. Shallow (commonly,) syntactic parsing 4. Resolution of co-reference among entities 5. Pattern-based event matching and role mapping 6. Normalization and output generation While accuracy at the lowest levels can reach high 90’s, as the stages advance, complexity increases and performance degrades considerably. The problem of IE as a whole, as well each of the listed subproblems, has been studied intensively for well over a decade, in many flavors and varieties. Key observations about much state-of-the-art IE are: a. IE is typically performed by a pipeline process; b. Only one hypothesis is propagated through the pipeline for each fact—the “best guess” the system can make for each slot fill; c. IE is performed in a document-by-document fashion, applying a priori knowledge locally to each document. The a priori knowledge may be encoded in a set of rules, an automatically trained model, or a hybrid thereof. Information extracted from documents— which may be termed a posteriori knowledge— is usually not reused across document boundaries, because the extracted facts are imprecise, and are therefore not a reliable basis for future extraction. Furthermore, locally non-best slot fills are not propagated through the pipeline, and are conse- quently not available downstream, nor for any global analysis. In most systems, these stages are performed in se- quence. The locally-best slot fills are passed from 57 the “lower-” to the “higher-level” modules, with- out feedback. Improvements are usually sought (e.g., the ACE research programme, (ACE, 2004)) by boosting performance at the lower levels, to reap benefits in the subsequent stages, where fewer errors are propagated. The point of departure for this paper is: the IE process is noisy and imprecise at the single- document level; this …

Content truncated to 500 words