Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4710–4723 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4710 Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs Deepak Nathani∗ Jatin Chauhan∗ Charu Sharma∗ Manohar Kaul Department of Computer Science and Engineering, IIT Hyderabad {deepakn1019,chauhanjatin100,charusharma1991}@gmail.com, mkaul@iith.ac.in Abstract The recent proliferation of knowledge graphs (KGs) coupled with incomplete or partial in- formation, in the form of missing relations (links) between entities, has fueled a lot of research on knowledge base completion (also known as relation prediction). Several re- cent works suggest that convolutional neural network (CNN) based models generate richer and more expressive feature embeddings and hence also perform well on relation prediction. However, we observe that these KG embed- dings treat triples independently and thus fail to cover the complex and hidden information that is inherently implicit in the local neighbor- hood surrounding a triple. To this effect, our paper proposes a novel attention-based feature embedding that captures both entity and rela- tion features in any given entity’s neighbor- hood. Additionally, we also encapsulate re- lation clusters and multi-hop relations in our model. Our empirical study offers insights into the efficacy of our attention-based model and we show marked performance gains in comparison to state-of-the-art methods on all datasets. 1 Introduction Knowledge graphs (KGs) represent knowledge bases (KBs) as a directed graph whose nodes and edges represent entities and relations between en- tities, respectively. For example, in Figure a triple (London, capital of, United Kingdom) is1, represented as two entities: London and United Kingdom along with a relation (capital of) linking them. KGs find uses in a wide variety of applica- tions such as semantic search (Berant et al., 2013; Berant and Liang, 2014), dialogue generation (He et al., 2017; Keizer et al., 2017), and question an- swering (Zhang et al., 2016; Diefenbach et al., 2018), to name a few. However, KGs typically ∗ Equal Contribution suffer from missing relations (Socher et al., 2013a; West et al., 2014). This problem gives rise to the task of knowledge base completion (also referred to as relation prediction), which entails predicting whether a given triple is valid or not. State-of-the-art relation prediction methods are known to be primarily knowledge embedding based models. They are broadly classified as translational models (Bordes et al., 2013; Yang et al., 2015; Trouillon et al., 2016) and convo- lutional neural network (CNN) (Nguyen et al., 2018; Dettmers et al., 2018) based models. While translational models learn embeddings using sim- ple operations and limited parameters, they pro- duce low quality embeddings. In contrast, CNN based models learn more expressive embeddings due to their parameter efficiency and consideration of complex relations. However, both translational and CNN based models process each triple inde- pendently and hence fail to encapsulate the seman- tically rich and latent relations that are inherently present in the vicinity of a given entity in a KG. Motivated by the aforementioned observa- tions, we propose a generalized attention-based graph …

Content truncated to 500 words