Incorporating Graph Attention Mechanism into Knowledge Graph Reasoning Based on Deep Reinforcement Learning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2623–2631, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2623 Incorporating Graph Attention Mechanism into Knowledge Graph Reasoning Based on Deep Reinforcement Learning Heng Wang⋆ Shuangyin Li∗ Rong Pan† Mingzhi Mao† ⋆Tencent, China †School of Data and Computer Science, Sun Yat-sen University, China ∗School of Computer Science, South China Normal University, China jimmyhwang@tencent.com {panr@mail,mcsmmz@mail}.sysu.edu.cn shuangyinli@m.scnu.edu.cn Abstract Knowledge Graph (KG) reasoning aims at finding reasoning paths for relations, in or- der to solve the problem of incompleteness in KG. Many previous path-based methods like PRA and DeepPath suffer from lacking mem- ory components, or stuck in training. There- fore, their performances always rely on well- pretraining. In this paper, we present a deep reinforcement learning based model named by AttnPath, which incorporates LSTM and Graph Attention Mechanism as the memory components. We define two metrics, Mean Se- lection Rate (MSR) and Mean Replacement Rate (MRR), to quantitatively measure how difficult it is to learn the query relations, and take advantages of them to fine-tune the model under the framework of reinforcement learn- ing. Meanwhile, a novel mechanism of re- inforcement learning is proposed by forcing an agent to walk forward every step to avoid the agent stalling at the same entity node con- stantly. Based on this operation, the proposed model not only can get rid of the pretraining process, but also achieves state-of-the-art per- formance comparing with the other models. We test our model on FB15K-237 and NELL- 995 datasets with different tasks. Extensive experiments show that our model is effective and competitive with many current state-of- the-art methods, and also performs well in practice. 1 Introduction Knowledge Graphs (KGs), such as NELL (Carl- son et al., 2010), Freebase (Bollacker et al., 2008) and WordNet (Miller, 1995) play an in- creasingly critical role in many downstream NLP applications, e.g., question answering (Dubey et al., 2018), information retrieval (Liu et al., 2018), personalized recommendations (Wang et al., 2019), etc. However, KGs are always incomplete, which would affect many down- stream tasks. Many links could miss among the entities in a KG. Thus, it is an important and challenge task on how to complete the KG by predicting the missing links between the entities through the method based on reasoning. For instance, if athletePlaysForTeam(X, Y ) and teamPlaysInLeague(Y, Z) both ex- ist in the KG, then we can infer that athletePlaysInLeague(X, Z), i.e, filling the missing edge athletePlaysInLeague between X and Z. There are mainly three ways to accomplish this task, such as Rule-Based (Wang and Cohen, 2016; Yang et al., 2017), Embedding-Based (Bor- des et al., 2013; Lin et al., 2015) and Path-Based (Lao et al., 2011). Meanwhile, it provides a new perspective to bring Deep Reinforcement Learn- ing (DRL) into the task of predicting the missing links, such as DeepPath (Xiong et al., 2017), a type of path-based method. DeepPath is the first work …

Content truncated to 500 words