Identifying Relations for Open Information Extraction

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1535–1545, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Identifying Relations for Open Information Extraction Anthony Fader, Stephen Soderland, and Oren Etzioni University of Washington, Seattle {afader,soderlan,etzioni}@cs.washington.edu Abstract Open Information Extraction (IE) is the task of extracting assertions from massive corpora without requiring a pre-specified vocabulary. This paper shows that the output of state-of- the-art Open IE systems is rife with uninfor- mative and incoherent extractions. To over- come these problems, we introduce two sim- ple syntactic and lexical constraints on bi- nary relations expressed by verbs. We im- plemented the constraints in the REVERB Open IE system, which more than doubles the area under the precision-recall curve relative to previous extractors such as TEXTRUNNER and WOEpos. More than 30% of REVERB’s extractions are at precision 0.8 or higher— compared to virtually none for earlier systems. The paper concludes with a detailed analysis of REVERB’s errors, suggesting directions for future work.1 1 Introduction and Motivation Typically, Information Extraction (IE) systems learn an extractor for each target relation from la- beled training examples (Kim and Moldovan, 1993; Riloff, 1996; Soderland, 1999). This approach to IE does not scale to corpora where the number of target relations is very large, or where the target relations cannot be specified in advance. Open IE solves this problem by identifying relation phrases—phrases that denote relations in English sentences (Banko et al., 2007). The automatic identification of rela- 1The source code for REVERB is available at http:// reverb.cs.washington.edu/ tion phrases enables the extraction of arbitrary re- lations from sentences, obviating the restriction to a pre-specified vocabulary. Open IE systems have achieved a notable measure of success on massive, open-domain corpora drawn from the Web, Wikipedia, and elsewhere. (Banko et al., 2007; Wu and Weld, 2010; Zhu et al., 2009). The output of Open IE systems has been used to support tasks like learning selectional preferences (Ritter et al., 2010), acquiring common sense knowledge (Lin et al., 2010), and recognizing entailment (Schoen- mackers et al., 2010; Berant et al., 2011). In ad- dition, Open IE extractions have been mapped onto existing ontologies (Soderland et al., 2010). We have observed that two types of errors are fre- quent in the output of Open IE systems such as TEX- TRUNNER and WOE: incoherent extractions and un- informative extractions. Incoherent extractions are cases where the ex- tracted relation phrase has no meaningful interpre- tation (see Table 1 for examples). Incoherent ex- tractions arise because the learned extractor makes a sequence of decisions about whether to include each word in the relation phrase, often resulting in incom- prehensible predictions. To solve this problem, we introduce a syntactic constraint: every multi-word relation phrase must begin with a verb, end with a preposition, and be a contiguous sequence of words in the sentence. Thus, the identification of a relation phrase is made in one fell swoop instead of on the basis of multiple, word-by-word decisions. Uninformative extractions …

Content truncated to 500 words