The Tradeoffs Between Open and Traditional Relation Extraction

Proceedings of ACL-08: HLT, pages 28–36, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics The Tradeoffs Between Open and Traditional Relation Extraction Michele Banko and Oren Etzioni Turing Center University of Washington Computer Science and Engineering Box 352350 Seattle, WA 98195, USA banko,etzioni@cs.washington.edu Abstract Traditional Information Extraction (IE) takes a relation name and hand-tagged examples of that relation as input. Open IE is a relation- independent extraction paradigm that is tai- lored to massive and heterogeneous corpora such as the Web. An Open IE system extracts a diverse set of relational tuples from text with- out any relation-specific input. How is Open IE possible? We analyze a sample of English sentences to demonstrate that numerous rela- tionships are expressed using a compact set of relation-independent lexico-syntactic pat- terns, which can be learned by an Open IE sys- tem. What are the tradeoffs between Open IE and traditional IE? We consider this question in the context of two tasks. First, when the number of relations is massive, and the rela- tions themselves are not pre-specified, we ar- gue that Open IE is necessary. We then present a new model for Open IE called O-CRF and show that it achieves increased precision and nearly double the recall than the model em- ployed by TEXTRUNNER, the previous state- of-the-art Open IE system. Second, when the number of target relations is small, and their names are known in advance, we show that O-CRF is able to match the precision of a tra- ditional extraction system, though at substan- tially lower recall. Finally, we show how to combine the two types of systems into a hy- brid that achieves higher precision than a tra- ditional extractor, with comparable recall. 1 Introduction Relation Extraction (RE) is the task of recognizing the assertion of a particular relationship between two or more entities in text. Typically, the target relation (e.g., seminar location) is given to the RE system as input along with hand-crafted extraction patterns or patterns learned from hand-labeled training exam- ples (Brin, 1998; Riloff and Jones, 1999; Agichtein and Gravano, 2000). Such inputs are specific to the target relation. Shifting to a new relation requires a person to manually create new extraction patterns or specify new training examples. This manual labor scales linearly with the number of target relations. In 2007, we introduced a new approach to the RE task, called Open Information Extraction (Open IE), which scales RE to the Web. An Open IE sys- tem extracts a diverse set of relational tuples without requiring any relation-specific human input. Open IE’s extraction process is linear in the number of documents in the corpus, and constant in the num- ber of relations. Open IE is ideally suited to corpora such as the Web, where the target relations are not known in advance, and their number is massive. The relationship between standard RE systems and the new Open IE paradigm is analogous to the relationship between lexicalized and unlexicalized parsers. Statistical parsers are usually lexicalized …

Content truncated to 500 words