Double-Branch Multi-Attention based Graph Neural Network for Knowledge Graph Completion

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 15257–15271 July 9-14, 2023 ©2023 Association for Computational Linguistics Double-Branch Multi-Attention based Graph Neural Network for Knowledge Graph Completion Hongcai Xu, Junpeng Bao ∗, Wenbo Liu Xi’an Jiaotong University xajt1822@stu.xjtu.edu.cn, baojp@mail.xjtu.edu.cn, stu_Lwb@stu.xjtu.edu.cn Abstract Graph neural networks (GNNs), which effec- tively use topological structures in the knowl- edge graphs (KG) to embed entities and rela- tions in low-dimensional spaces, have shown great power in knowledge graph completion (KGC). KG has abundant global and local struc- tural information, however, many GNN-based KGC models cannot capture these two types of information about the graph structure by de- signing complex aggregation schemes and are not designed well to learn representations of seen entities with sparse neighborhoods in iso- lated subgraphs. In this paper, we find that a simple attention-based method can outperform a general GNN-based approach for KGC. We then propose a double-branch multi-attention- based graph neural network (MA-GNN) to learn more expressive entity representations that contain rich global-local structural infor- mation. Specifically, we first explore the graph attention network-based local aggregator to learn entity representations. Furthermore, we propose a snowball local attention mechanism by leveraging the semantic similarity between two-hop neighbors to enrich the entity embed- ding. Finally, we use Transformer-based self- attention to learn long-range dependence be- tween entities to obtain richer representations with the global graph structure and entity fea- tures. Experimental results on five benchmark datasets show that MA-GNN achieves signif- icant improvements over strong baselines for inductive KGC. 1 Introduction Knowledge graphs (KGs), which are collections of structured knowledge represented by factual triples (head entity, relation, tail entity), are crucial for many applications, including semantic search (Xiong et al., 2017), question-answering (Kaiser et al., 2021), recommendation systems (Wang et al., 2021), etc. However, even large-scale KGs (e.g., ∗The corresponding author. Freebase and DBpedia) that have billions of triples, are inevitably incomplete, limiting their real-world applications. Therefore, knowledge graph comple- tion (KGC) recently attracted extensive attention and attempts to automatically predict the missing head (tail) given the relation and tail (head) in a factual triple. To address these challenges, knowledge graph embedding (KGE) methods have been recently in- troduced as an effective solution to solve the in- completion problem (Bordes et al., 2013; Dettmers et al., 2018; Sun et al., 2019; Trouillon et al., 2016). KGE aims to define a scoring function and embeds entities and relations into a low-dimensional vector space to assess the plausibility of triplets based on the observed triples. However, it is significantly re- liant on the pre-defined scoring function and rather challenging to encode structural information about an entity into a single vector. To this end, graph neural networks (GNNs) (Def- ferrard et al., 2016; Kipf and Welling, 2017) have recently been proposed for inductive KGC, due to the intrinsic graph structure of KGs, which learn the hidden representation of each entity by ag- gregating its corresponding local neighbors’ infor- mation (Hamaguchi et al., 2017; Albooyeh et al., …

Content truncated to 500 words