Path Spuriousness-aware Reinforcement Learning for Multi-Hop Knowledge Graph Reasoning

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3181–3192 May 2-6, 2023 ©2023 Association for Computational Linguistics Path Spuriousness-aware Reinforcement Learning for Multi-Hop Knowledge Graph Reasoning Chunyang Jiang1,2, Tianchen Zhu1,2,4, Haoyi Zhou1,3, Chang Liu3, Ting Deng1,2, Chunming Hu1,3 and Jianxin Li1,2� 1SKLSDE, Beihang University 2School of Computer Science and Engineering, Beihang University 3School of Software, Beihang University 4Shenyuan Honors College, Beihang University {jiangcy,zhutc,zhouhy,liuc,dengting,hucm,lijx}@act.buaa.edu.cn Abstract Multi-hop reasoning, a prevalent approach for query answering, aims at inferring new facts along reasonable paths over a knowledge graph. Reinforcement learning (RL) meth- ods can be adopted by formulating the prob- lem into a Markov decision process. How- ever, common suffering within RL-based rea- soning models is that the agent can be biased to spurious paths which coincidentally lead to the correct answer with poor explanation. In this work, we take a deep dive into this phe- nomenon and define a metric named Path Spu- riousness (PS), to quantitatively estimate to what extent a path is spurious. Guided by the definition of PS, we design a model with a new reward that considers both answer accuracy and path reasonableness. We test our method on five datasets and experiments reveal that our method considerably enhances the agent’s ca- pacity to prevent spurious paths while keeping comparable to state-of-the-art performance. 1 Introduction Knowledge Graph (KG), a set of structured facts about real-world human knowledge, is utilized in numerous downstream NLP applications (Hilde- brandt et al., 2020; Zhang and Yao, 2022; Xu et al., 2021; Ma et al., 2021). Common suffering affect- ing many downstream tasks is KG incompleteness. A variable amount of facts are missing in practi- cal KGs. KG reasoning, the process to derive new knowledge from KG (Ji et al., 2022; Zhang et al., 2022; Huang et al., 2022), is the way to address KG completion problem. A prevailing approach for KG reasoning is incorporating KG embedding (KGE), which maps entities and relations into a vec- tor space (Bordes et al., 2013). Embedding-based models have great power in expressing semantic similarity of entities and relations, but usually lack explainability due to the high-dimension represen- tation (Chen et al., 2020; Heo et al., 2022). � The corresponding author is Jianxin Li. Tom MIT Mary Dave John Jerry U.S. Alice Stanford CMU UC Berkeley Bob teach teach workAt workAt beFriends beFriends educatedAt educatedAt bornIn bornIn bornIn visit visit visit educatedAt? H.S. educatedAt Figure 1: An example KG (we use H.S. to indicate a certain high school for short). Supposing the query is <Tom, educatedAt, ?> and MIT is a correct answer, three paths leading to MIT are of different PS. An alternative approach is multi-hop reasoning, also referred to as path-based models (Lin et al., 2018), which infers new facts along existing paths in KG. Multi-hop reasoning offers explanations for its predictions by taking advantage of reasoning paths. Recently, reinforcement learning (RL) has been applied to multi-hop reasoning (Xiong et al., 2017). RL-based methods train an agent to walk …

Content truncated to 500 words