Generalizing to Unseen Entities and Entity Pairs with Row-less Universal Schema

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 613–622, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Generalizing to Unseen Entities and Entity Pairs with Row-less Universal Schema Patrick Verga, Arvind Neelakantan and Andrew McCallum College of Information and Computer Sciences University of Massachusetts Amherst {pat, arvind, mccallum}@cs.umass.edu Abstract Universal schema predicts the types of enti- ties and relations in a knowledge base (KB) by jointly embedding the union of all avail- able schema types—not only types from mul- tiple structured databases (such as Freebase or Wikipedia infoboxes), but also types ex- pressed as textual patterns from raw text. This prediction is typically modeled as a matrix completion problem, with one type per col- umn, and either one or two entities per row (in the case of entity types or binary relation types, respectively). Factorizing this sparsely observed matrix yields a learned vector em- bedding for each row and each column. In this paper we explore the problem of making pre- dictions for entities or entity-pairs unseen at training time (and hence without a pre-learned row embedding). We propose an approach having no per-row parameters at all; rather we produce a row vector on the fly using a learned aggregation function of the vectors of the observed columns for that row. We exper- iment with various aggregation functions, in- cluding neural network attention models. Our approach can be understood as a natural lan- guage database, in that questions about KB entities are answered by attending to textual or database evidence. In experiments predict- ing both relations and entity types, we demon- strate that despite having an order of magni- tude fewer parameters than traditional univer- sal schema, we can match the accuracy of the traditional model, and more importantly, we can now make predictions about unseen rows with nearly the same accuracy as rows avail- able at training time. 1 Introduction Automatic knowledge base construction (AKBC) is the task of building a structured knowledge base (KB) of facts using raw text evidence, and often an initial seed KB to be augmented (Carlson et al., 2010; Suchanek et al., 2007; Bollacker et al., 2008). KBs generally con- tain entity type facts such as Sundar Pichai IsA Per- son and relation facts such as CEO Of(Sundar Pichai, Google). Extracted facts about entities, and their types and relations are useful for many downstream tasks such as question answering (Bordes et al., 2014) and semantic parsing (Berant et al., 2013; Kwiatkowski et al., 2013). An effective approach to AKBC is universal schema, which predicts the types of entities and relations in a knowledge base (KB) by jointly embedding the union of all available schema types—not only types from multiple structured databases (such as Freebase or Wikipedia infoboxes), but also types expressed as tex- tual patterns from raw text. This prediction is typically modeled as a matrix completion problem. In the stan- dard formulation for relation extraction (Riedel et al., 2013), …

Content truncated to 500 words