Using Corpus Statistics on Entities to Improve Semi-supervised Relation Extraction from the Web

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 600–607, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Using Corpus Statistics on Entities to Improve Semi-supervised Relation Extraction from the Web Benjamin Rosenfeld Information Systems HU School of Business, Hebrew University, Jerusalem, Israel grurgrur@gmail.com Ronen Feldman Information Systems HU School of Business, Hebrew University, Jerusalem, Israel ronen.feldman@huji.ac.il Abstract Many errors produced by unsupervised and semi-supervised relation extraction (RE) systems occur because of wrong recogni- tion of entities that participate in the rela- tions. This is especially true for systems that do not use separate named-entity rec- ognition components, instead relying on general-purpose shallow parsing. Such sys- tems have greater applicability, because they are able to extract relations that contain attributes of unknown types. However, this generality comes with the cost in accuracy. In this paper we show how to use corpus statistics to validate and correct the arguments of extracted relation instances, improving the overall RE performance. We test the methods on SRES – a self-supervised Web relation extraction system. We also compare the performance of corpus-based methods to the performance of validation and correc- tion methods based on supervised NER components. 1 Introduction Information Extraction (IE) is the task of extract- ing factual assertions from text. Most IE systems rely on knowledge engineering or on machine learning to generate the “task model” that is subse- quently used for extracting instances of entities and relations from new text. In the knowledge engi- neering approach the model (usually in the form of extraction rules) is created manually, and in the machine learning approach the model is learned automatically from a manually labeled training set of documents. Both approaches require substantial human effort, particularly when applied to the broad range of documents, entities, and relations on the Web. In order to minimize the manual ef- fort necessary to build Web IE systems, semi- supervised and completely unsupervised systems are being developed by many researchers. The task of extracting facts from the Web has significantly different aims than the regular infor- mation extraction. The goal of regular IE is to identify and label all mentions of all instances of the given relation type inside a document or inside a collection of documents. Whereas, in the Web Extraction (WE) tasks we are only interested in extracting relation instances and not interested in particular mentions. This difference in goals leads to a difference in the methods of performance evaluation. The usual measures of performance of regular IE systems are precision, recall, and their combinations – the breakeven point and F-measure. Unfortunately, the true recall usually cannot be known for WE tasks. Consequently, for evaluating the performance of WE systems, the recall is substituted by the num- ber of extracted instances. WE systems usually order the extracted in- stances by the system’s confidence in their cor- rectness. The precision of top-confidence extrac- tions is usually very high, but it gets progressively lower when lower-confidence candidates are con- sidered. The curve …

Content truncated to 500 words