A2N: Attending to Neighbors for Knowledge Graph Inference

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4387–4392 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4387 A2N: Attending to Neighbors for Knowledge Graph Inference Trapit Bansal†∗and Da-Cheng Juan‡ and Sujith Ravi‡ and Andrew McCallum† † University of Massachusetts, Amherst ‡ Google Research {tbansal, mccallum}@cs.umass.edu {dacheng, sravi}@google.com Abstract State-of-the-art models for knowledge graph completion aim at learning a fixed embed- ding representation of entities in a multi- relational graph which can generalize to in- fer unseen entity relationships at test time. This can be sub-optimal as it requires mem- orizing and generalizing to all possible en- tity relationships using these fixed represen- tations. We thus propose a novel attention- based method to learn query-dependent rep- resentation of entities which adaptively com- bines the relevant graph neighborhood of an entity leading to more accurate KG comple- tion. The proposed method is evaluated on two benchmark datasets for knowledge graph com- pletion, and experimental results show that the proposed model performs competitively or better than existing state-of-the-art, including recent methods for explicit multi-hop reason- ing. Qualitative probing offers insight into how the model can reason about facts involv- ing multiple hops in the knowledge graph, through the use of neighborhood attention. 1 Introduction Knowledge graphs, such as Freebase (Bollacker et al., 2008), contain a wealth of structured knowl- edge in the form of relationships between enti- ties and are useful for numerous end applications. However, knowledge graphs (KG)—whether au- tomatically constructed or human curated—are in- complete (Banko et al., 2007) and thus automatic methods for KG completion have been an impor- tant area of research (Nickel et al., 2016). The task of KG completion requires inferring missing en- tity relationships from the observed graph and is often formulated as predicting a target entity for a given query of source entity e and relation r, that is, to complete the tuple (e, r, ?). ∗Work done as an intern at Google Research Most state-of-the-art methods for KG comple- tion learn vector embeddings of entities and rela- tions (Bordes et al., 2013; Toutanova et al., 2015; Dettmers et al., 2017; Trouillon et al., 2016) which are used in conjunction with a (potentially param- eterized) scoring function that scores every tuple in the graph. These embeddings are optimized such that the score for observed graph tuples is higher than a random tuple. While these mod- els achieve good performance, they learn a fixed- dimensional embedding for every entity, which necessitates that this embedding must memorize and then be able to generalize to infer all possi- ble relationships for the entity, which may require multiple-hops of reasoning in the KG (Neelakan- tan et al., 2015; Das et al., 2017). In contrast, it can be beneficial to compose em- beddings from a query-relevant subset of the graph neighborhood of the entity. As a motivating exam- ple, consider answering the query (e, nationality, ?) for some entity e. Observing the KG neighbor (e, lived in, Maui), can …

Content truncated to 500 words