Compositional Vector Space Models for Knowledge Base Completion

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 156–166, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Compositional Vector Space Models for Knowledge Base Completion Arvind Neelakantan, Benjamin Roth, Andrew McCallum Department of Computer Science University of Massachusetts, Amherst Amherst, MA, 01003 {arvind,beroth,mccallum}@cs.umass.edu Abstract Knowledge base (KB) completion adds new facts to a KB by making inferences from existing facts, for example by infer- ring with high likelihood nationality(X,Y) from bornIn(X,Y). Most previous methods infer simple one-hop relational synonyms like this, or use as evidence a multi-hop re- lational path treated as an atomic feature, like bornIn(X,Z) → containedIn(Z,Y). This paper presents an approach that reasons about conjunctions of multi-hop relations non-atomically, composing the implica- tions of a path using a recurrent neural network (RNN) that takes as inputs vec- tor embeddings of the binary relation in the path. Not only does this allow us to generalize to paths unseen at training time, but also, with a single high-capacity RNN, to predict new relation types not seen when the compositional model was trained (zero-shot learning). We assem- ble a new dataset of over 52M relational triples, and show that our method im- proves over a traditional classifier by 11%, and a method leveraging pre-trained em- beddings by 7%. 1 Introduction Constructing large knowledge bases (KBs) sup- ports downstream reasoning about resolved enti- ties and their relations, rather than the noisy tex- tual evidence surrounding their natural language mentions. For this reason KBs have been of in- creasing interest in both industry and academia (Bollacker et al., 2008; Suchanek et al., 2007; Carlson et al., 2010). Such KBs typically con- tain many millions of facts, most of them (en- tity1,relation,entity2) “triples” (also known as bi- nary relations) such as (Barack Obama, presi- dentOf, USA) and (Brad Pitt, marriedTo, Angelina Jolie). However, even the largest KBs are woefully in- complete (Min et al., 2013), missing many impor- tant facts, and therefore damaging their usefulness in downstream tasks. Ironically, these missing facts can frequently be inferred from other facts al- ready in the KB, thus representing a sort of incon- sistency that can be repaired by the application of an automated process. The addition of new triples by leveraging existing triples is typically known as KB completion. Early work on this problem focused on learn- ing symbolic rules. For example, Schoenmack- ers et al. (2010) learns Horn clauses predictive of new binary relations by exhausitively exploring re- lational paths of increasing length, and selecting those surpassing an accuracy threshold. (A “path” is a sequence of triples in which the second entity of each triple matches the first entity of the next triple.) Lao et al. (2011) introduced the Path Rank- ing Algorithm (PRA), which greatly improves ef- ficiency and robustness by replacing exhaustive search with random walks, and using unique paths as features in a per-target-relation binary classifier. A typical predictive feature learned by PRA is …

Content truncated to 500 words