Learning 5000 Relational Extractors

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 286–295, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Learning 5000 Relational Extractors Raphael Hoffmann, Congle Zhang, Daniel S. Weld Computer Science & Engineering University of Washington Seattle, WA-98195, USA {raphaelh,clzhang,weld}@cs.washington.edu Abstract Many researchers are trying to use information extraction (IE) to create large-scale knowl- edge bases from natural language text on the Web. However, the primary approach (su- pervised learning of relation-specific extrac- tors) requires manually-labeled training data for each relation and doesn’t scale to the thou- sands of relations encoded in Web text. This paper presents LUCHS, a self-supervised, relation-specific IE system which learns 5025 relations — more than an order of magnitude greater than any previous approach — with an average F1 score of 61%. Crucial to LUCHS’s performance is an automated system for dy- namic lexicon learning, which allows it to learn accurately from heuristically-generated training data, which is often noisy and sparse. 1 Introduction Information extraction (IE), the process of gen- erating relational data from natural-language text, has gained popularity for its potential applications in Web search, question answering and other tasks. Two main approaches have been attempted: • Supervised learning of relation-specific ex- tractors (e.g., (Freitag, 1998)), and • “Open” IE — self-supervised learning of unlexicalized, relation-independent extractors (e.g., Textrunner (Banko et al., 2007)). Unfortunately, both methods have problems. Supervised approaches require manually-labeled training data for each relation and hence can’t scale to handle the thousands of relations encoded in Web text. Open extraction is more scalable, but has lower precision and recall. Furthermore, open extraction doesn’t canonicalize relations, so any application using the output must deal with homonymy and synonymy. A third approach, sometimes refered to as weak supervision, is to heuristically match values from a database to text, thus generating a set of train- ing data for self-supervised learning of relation- specific extractors (Craven and Kumlien, 1999). With the Kylin system (Wu and Weld, 2007) ap- plied this idea to Wikipedia by matching values of an article’s infobox1 attributes to corresponding sentences in the article, and suggested that their approach could extract thousands of relations (Wu et al., 2008). Unfortunately, however, they never tested the idea on more than a dozen relations. In- deed, no one has demonstrated a practical way to extract more than about one hundred relations. We note that Wikipedia’s infobox ‘ontology’ is a particularly interesting target for extraction. As a by-product of thousands of contributors, it is broad in coverage and growing quickly. Unfortunately, the schemata are surprisingly noisy and most are sparsely populated; challenging conditions for ex- traction. This paper presents LUCHS, an autonomous, self-supervised system, which learns 5025 rela- tional extractors — an order of magnitude greater than any previous effort. Like Kylin, LUCHS cre- ates training data by matching Wikipedia attribute values with corresponding sentences, but by itself, this method was insufficient for accurate extrac- tion of most relations. Thus, LUCHS introduces a new technique, dynamic lexicon features, which dramatically …

Content truncated to 500 words