Learning Collaborative Agents with Rule Guidance for Knowledge Graph Reasoning

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8541–8547, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8541 Learning Collaborative Agents with Rule Guidance for Knowledge Graph Reasoning Deren Lei1∗, Gangrong Jiang1∗, Xiaotao Gu2, Kexuan Sun1, Yuning Mao2, Xiang Ren1 1University of Southern California 2University of Illinois at Urbana-Champaign {derenlei, gjiang, kexuansu, xiangren}@usc.edu, {xiaotao2, yuningm2}@illinois.edu Abstract Walk-based models have shown their advan- tages in knowledge graph (KG) reasoning by achieving decent performance while providing interpretable decisions. However, the sparse reward signals offered by the KG during traver- sal are often insufficient to guide a sophisti- cated walk-based reinforcement learning (RL) model. An alternate approach is to use tra- ditional symbolic methods (e.g., rule induc- tion), which achieve good performance but can be hard to generalize due to the lim- itation of symbolic representation. In this paper, we propose RuleGuider, which lever- ages high-quality rules generated by symbolic- based methods to provide reward supervision for walk-based agents. Experiments on bench- mark datasets show that RuleGuider improves the performance of walk-based models with- out losing interpretability. 1 1 Introduction While knowledge graphs (KGs) are widely adopted in natural language processing applications, a ma- jor bottleneck hindering its usage is the sparsity of facts (Min et al., 2013), leading to extensive studies on KG completion (or reasoning) (Trouillon et al., 2016; Dettmers et al., 2018; Das et al., 2017; Xiong et al., 2017; Lin et al., 2018; Meilicke et al., 2019). Many traditional approaches on the KG reason- ing task are based on logic rules (Landwehr et al., 2007, 2010; Gal´arraga et al., 2013, 2015). These methods are referred to as symbolic-based methods. Although they showed good performance (Meil- icke et al., 2019, 2020), they are inherently limited by their representations and generalizability of the associated relations of the given rules. To ameliorate such limitations, embedding- based methods (Bordes et al., 2013; Socher et al., ∗Equal contributions. 1https://github.com/derenlei/ KG-RuleGuider 2013; Wang et al., 2014; Yang et al., 2014; Trouil- lon et al., 2016; Dettmers et al., 2018, 2017; Sun et al., 2019; Zhang et al., 2019) were proposed. They learn distributed representations for entities and relations and make predictions using the rep- resentations. Despite their superior performance, they fail to make human-friendly interpretations. To improve the interpretability, many recent ef- forts formulate the task as a multi-hop reasoning problem using reinforcement learning (RL) tech- niques (Xiong et al., 2017; Das et al., 2017; Shen et al., 2018; Chen et al., 2018; Lin et al., 2018), re- ferred to as walk-based methods. A major issue of these methods is the reward function. A “hit or not” reward is too sparse while a shaped reward using an embedding-based distance measurement Lin et al. (2018) may not always result in desirable paths. In this paper, we propose RuleGuider to tackle the aforementioned reward issue in walk-based methods with the help of symbolic rules. We aim to improve the performance of walk-based methods without losing their interpretability. The RuleGuider is composed of …

Content truncated to 500 words