Espresso: Leveraging Generic Patterns for Automatically Harvesting Semantic Relations

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 113–120, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Espresso: Leveraging Generic Patterns for Automatically Harvesting Semantic Relations Patrick Pantel Information Sciences Institute University of Southern California 4676 Admiralty Way Marina del Rey, CA 90292 pantel@isi.edu Marco Pennacchiotti ART Group - DISP University of Rome “Tor Vergata” Viale del Politecnico 1 Rome, Italy pennacchiotti@info.uniroma2.it Abstract In this paper, we present Espresso, a weakly-supervised, general-purpose, and accurate algorithm for harvesting semantic relations. The main contribu- tions are: i) a method for exploiting ge- neric patterns by filtering incorrect instances using the Web; and ii) a prin- cipled measure of pattern and instance reliability enabling the filtering algo- rithm. We present an empirical com- parison of Espresso with various state of the art systems, on different size and genre corpora, on extracting various general and specific relations. Experi- mental results show that our exploita- tion of generic patterns substantially increases system recall with small effect on overall precision. 1 Introduction Recent attention to knowledge-rich problems such as question answering (Pasca and Harabagiu 2001) and textual entailment (Geffet and Dagan 2005) has encouraged natural language process- ing researchers to develop algorithms for auto- matically harvesting shallow semantic resources. With seemingly endless amounts of textual data at our disposal, we have a tremendous opportu- nity to automatically grow semantic term banks and ontological resources. To date, researchers have harvested, with varying success, several resources, including concept lists (Lin and Pantel 2002), topic signa- tures (Lin and Hovy 2000), facts (Etzioni et al. 2005), and word similarity lists (Hindle 1990). Many recent efforts have also focused on extract- ing semantic relations between entities, such as entailments (Szpektor et al. 2004), is-a (Ravi- chandran and Hovy 2002), part-of (Girju et al. 2006), and other relations. The following desiderata outline the properties of an ideal relation harvesting algorithm: • Performance: it must generate both high preci- sion and high recall relation instances; • Minimal supervision: it must require little or no human annotation; • Breadth: it must be applicable to varying cor- pus sizes and domains; and • Generality: it must be applicable to a wide va- riety of relations (i.e., not just is-a or part-of). To our knowledge, no previous harvesting algo- rithm addresses all these properties concurrently. In this paper, we present Espresso, a general- purpose, broad, and accurate corpus harvesting algorithm requiring minimal supervision. The main algorithmic contribution is a novel method for exploiting generic patterns, which are broad coverage noisy patterns – i.e., patterns with high recall and low precision. Insofar, difficulties in using these patterns have been a major impedi- ment for minimally supervised algorithms result- ing in either very low precision or recall. We propose a method to automatically detect generic patterns and to separate their correct and incor- rect instances. The key intuition behind the algo- rithm is that given a set of reliable (high precision) patterns on a corpus, correct instances of …

Content truncated to 500 words