Relation Extraction with Matrix Factorization and Universal Schemas

Proceedings of NAACL-HLT 2013, pages 74–84, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Relation Extraction with Matrix Factorization and Universal Schemas Sebastian Riedel Department of Computer Science University College London s.riedel@ucl.ac.uk Limin Yao, Andrew McCallum, Benjamin M. Marlin Department of Computer Science University of Massachusetts at Amherst {lmyao,mccallum,marlin}@cs.umass.edu Abstract Traditional relation extraction predicts rela- tions within some fixed and finite target schema. Machine learning approaches to this task require either manual annotation or, in the case of distant supervision, existing struc- tured sources of the same schema. The need for existing datasets can be avoided by us- ing a universal schema: the union of all in- volved schemas (surface form predicates as in OpenIE, and relations in the schemas of pre- existing databases). This schema has an al- most unlimited set of relations (due to surface forms), and supports integration with existing structured data (through the relation types of existing databases). To populate a database of such schema we present matrix factorization models that learn latent feature vectors for en- tity tuples and relations. We show that such latent models achieve substantially higher ac- curacy than a traditional classification ap- proach. More importantly, by operating simul- taneously on relations observed in text and in pre-existing structured DBs such as Freebase, we are able to reason about unstructured and structured data in mutually-supporting ways. By doing so our approach outperforms state- of-the-art distant supervision. 1 Introduction Most previous work in relation extraction uses a pre- defined, finite and fixed schema of relation types (such as born-in or employed-by). Usually some tex- tual data is labeled according to this schema, and this labeling is then used in supervised training of an automated relation extractor, e.g. Culotta and Sorensen (2004). However, labeling textual rela- tions is time-consuming and difficult, leading to sig- nificant recent interest in distantly-supervised learn- ing. Here one aligns existing database records with the sentences in which these records have been “ren- dered”––effectively labeling the text—and from this labeling we can train a machine learning system as before (Craven and Kumlien, 1999; Mintz et al., 2009; Bunescu and Mooney, 2007; Riedel et al., 2010). However, this method relies on the availabil- ity of a large database that has the desired schema. The need for pre-existing datasets can be avoided by using language itself as the source of the schema. This is the approach taken by OpenIE (Etzioni et al., 2008). Here surface patterns between mentions of concepts serve as relations. This approach requires no supervision and has tremendous flexibility, but lacks the ability to generalize. For example, Ope- nIE may find FERGUSON–historian-at–HARVARD but does not know FERGUSON–is-a-professor-at– HARVARD. OpenIE has traditionally relied on a large diversity of textual expressions to provide good coverage. But this diversity is not always available, and, in any case, the lack of generalization greatly inhibits the ability to support reasoning. One way to gain generalization is to cluster tex- tual surface forms that have similar meaning (Lin and Pantel, 2001; Pantel et …

Content truncated to 500 words