Injecting Logical Background Knowledge into Embeddings for Relation Extraction

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 1119–1129, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Injecting Logical Background Knowledge into Embeddings for Relation Extraction Tim Rockt¨aschel University College London London, UK Sameer Singh University of Washington Seattle, WA Sebastian Riedel University College London London, UK Abstract Matrix factorization approaches to relation extraction provide several attractive features: they support distant supervision, handle open schemas, and leverage unlabeled data. Unfortu- nately, these methods share a shortcoming with all other distantly supervised approaches: they cannot learn to extract target relations with- out existing data in the knowledge base, and likewise, these models are inaccurate for rela- tions with sparse data. Rule-based extractors, on the other hand, can be easily extended to novel relations and improved for existing but inaccurate relations, through first-order formu- lae that capture auxiliary domain knowledge. However, usually a large set of such formulae is necessary to achieve generalization. In this paper, we introduce a paradigm for learning low-dimensional embeddings of entity-pairs and relations that combine the ad- vantages of matrix factorization with first-order logic domain knowledge. We introduce simple approaches for estimating such embeddings, as well as a novel training algorithm to jointly optimize over factual and first-order logic in- formation. Our results show that this method is able to learn accurate extractors with little or no distant supervision alignments, while at the same time generalizing to textual patterns that do not appear in the formulae. 1 Introduction Relation extraction, the task of identifying rela- tions between named entities, is a crucial compo- nent for information extraction. A recent successful approach (Riedel et al., 2013) relies on two ideas: (a) unifying traditional canonical relations, such as those of the Freebase schema, with OpenIE surface form patterns in a universal schema, and (b) complet- ing a knowledge base of such a schema using matrix factorization. This approach has several attractive properties. First, for canonical relations it effectively performs distant supervision (Bunescu and Mooney, 2007; Mintz et al., 2009; Yao et al., 2011; Hoffmann et al., 2011; Surdeanu et al., 2012) and hence re- quires no textual annotations. Second, in the spirit of OpenIE, a universal schema can use textual patterns as novel relations and thus increases the coverage of traditional schemas (Riedel et al., 2013; Fan et al., 2014). Third, matrix factorization learns better embeddings for entity-pairs for which only surface form patterns are observed, and these can also lead to better extractions of canonical relations. Unfortunately, populating a universal schema knowledge base using matrix factorization suffers from a problem all distantly-supervised techniques share: you can only reliably learn relations that ap- pear frequently enough in the knowledge base. In par- ticular, for relations that do not appear in the knowl- edge base or for which no facts are known we cannot learn a predictor at all. One way to overcome this problem is to incorporate additional domain knowl- edge, either specified manually or bootstrapped …

Content truncated to 500 words