Passage Retrieval for Information Extraction using Distant Supervision

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 1046–1054, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Passage Retrieval for Information Extraction using Distant Supervision Wei Xu° Ralph Grishman° Le Zhao* °New York University New York, NY, USA xuwei,grishman@cs.nyu.edu *Carnegie Mellon University Pittsburgh, PA, USA lezhao@cs.cmu.edu Abstract In this paper, we propose a keyword-based passage retrieval algorithm for information ex- traction, trained by distant supervision. Our goal is to be able to extract attributes of people and organizations more quickly and accurately by first ranking all the potentially relevant passages according to their likelihood of con- taining the answer and then performing a tradi- tional deeper, slower analysis of individual passages. Using Freebase as our source of known relation instances and Wikipedia as our text source, we collected a weighted set of keywords indicative of each relation and then use it to re-rank the passages retrieved by the Lemur search engine. Experiments show that our algorithm significantly outperforms state- of-the-art passage retrieval techniques in eval- uations of both individual passage retrieval and end-to-end information extraction. 1 Introduction Large-corpus information extraction involves the extraction of pre-specified types of relations and events from large corpora. For example, the Knowledge Base Population (KBP) slot-filling task (Ji et al., 2010) involves finding, from a large corpus, a few dozen attributes of a speci- fied person or organization. In many cases we do not have the time to per- form in-depth extraction for all attributes over the entire corpus. Consequently, addressing this task typically involves a blend of traditional question answering (QA) and information extrac- tion (IE) methods. Like QA, we need to begin with passage retrieval, where a passage can range from a sentence to a piece of text or a document. However, unlike QA, we have a fixed inventory of relations and a fixed set of expected answer types (e.g. employer of a person). This allows us to bring to bear the more specialized learning methods of IE to tune the passage retrieval for each relation of interest. To the best of our knowledge, we are the first to systematically study the passage retrieval al- gorithm for information extraction and propose a novel distant supervision approach to obtain a list of weighted keywords for each relation. Dis- tant supervision (Mintz et al., 2009) makes use of noisy training data generated automatically from a related, but different, type dataset to solve problems on another type of data. Instead of a handful of human-selected keywords, we auto- matically learn hundreds or thousands of indica- tive keywords from a freely available online re- source, Freebase, which is similar to Wikipedia Infoboxes. Passages are ranked and retrieved based on these keywords indicative of certain relations. We then feed individual passages to a traditional IE system or to an answer extraction component as used in QA systems to obtain the final outputs. Both the training and testing pro- cedures of our method require only statistics of surface words and named entities in …

Content truncated to 500 words