Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 541–550, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, Daniel S. Weld Computer Science & Engineering University of Washington Seattle, WA 98195, USA {raphaelh,clzhang,xiaoling,lsz,weld}@cs.washington.edu Abstract Information extraction (IE) holds the promise of generating a large-scale knowledge base from the Web’s natural language text. Knowledge-based weak supervision, using structured data to heuristically label a training corpus, works towards this goal by enabling the automated learning of a potentially unbounded number of relation extractors. Recently, researchers have developed multi- instance learning algorithms to combat the noisy training data that can come from heuristic labeling, but their models assume relations are disjoint — for example they cannot extract the pair Founded(Jobs, Apple) and CEO-of(Jobs, Apple). This paper presents a novel approach for multi-instance learning with overlapping re- lations that combines a sentence-level extrac- tion model with a simple, corpus-level compo- nent for aggregating the individual facts. We apply our model to learn extractors for NY Times text using weak supervision from Free- base. Experiments show that the approach runs quickly and yields surprising gains in accuracy, at both the aggregate and sentence level. 1 Introduction Information-extraction (IE), the process of generat- ing relational data from natural-language text, con- tinues to gain attention. Many researchers dream of creating a large repository of high-quality extracted tuples, arguing that such a knowledge base could benefit many important tasks such as question an- swering and summarization. Most approaches to IE use supervised learning of relation-specific exam- ples, which can achieve high precision and recall. Unfortunately, however, fully supervised methods are limited by the availability of training data and are unlikely to scale to the thousands of relations found on the Web. A more promising approach, often called “weak” or “distant” supervision, creates its own training data by heuristically matching the contents of a database to corresponding text (Craven and Kum- lien, 1999). For example, suppose that r(e1, e2) = Founded(Jobs, Apple) is a ground tuple in the database and s =“Steve Jobs founded Apple, Inc.” is a sentence containing synonyms for both e1 = Jobs and e2 = Apple, then s may be a natural language expression of the fact that r(e1, e2) holds and could be a useful training example. While weak supervision works well when the tex- tual corpus is tightly aligned to the database con- tents (e.g., matching Wikipedia infoboxes to as- sociated articles (Hoffmann et al., 2010)), Riedel et al. (2010) observe that the heuristic leads to noisy data and poor extraction performance when the method is applied more broadly (e.g., matching Freebase records to NY Times articles). To fix this problem they cast weak supervision as a form of multi-instance learning, assuming only that at least one of the sentences containing e1 and e2 are ex- pressing r(e1, e2), and their method yields a sub- stantial improvement in extraction performance. …

Content truncated to 500 words