Ruleformer: Context-aware Rule Mining over Knowledge Graph

Proceedings of the 29th International Conference on Computational Linguistics, pages 2551–2560 October 12–17, 2022. 2551 Ruleformer: Context-aware Rule Mining over Knowledge Graph Zezhong Xu1 , Peng Ye1 , Hui Chen3, Meng Zhao4 Huajun Chen1,2, Wen Zhang1∗ 1Zhejiang University & AZFT Joint Lab for Knowledge Engine, China 2Hangzhou Innovation Center, Zhejiang University 3Alibaba Group, 4Huawei {xuzezhong, yep, huajunsir, zhang.wen}@zju.edu.cn, weidu.ch@alibaba-inc.com, zhaomeng57@huawei.com Abstract Rule mining is an effective approach for rea- soning over knowledge graph (KG). Existing works mainly concentrate on mining rules. However, there might be several rules that could be applied for reasoning for one relation, and how to select appropriate rules for comple- tion of different triples has not been discussed. In this paper, we propose to take the context information into consideration, which helps se- lect suitable rules for the inference tasks. Based on this idea, we propose a transformer-based rule mining approach, Ruleformer1. It consists of two blocks: 1) an encoder extracting the con- text information from subgraph of head entities with modified attention mechanism, and 2) a decoder which aggregates the subgraph infor- mation from the encoder output and generates the probability of relations for each step of rea- soning. The basic idea behind Ruleformer is regarding rule mining process as a sequence to sequence task. To make the subgraph a se- quence input to the encoder and retain the graph structure, we devise a relational attention mech- anism in Transformer. The experiment results show the necessity of considering these infor- mation in rule mining task and the effectiveness of our model. 1 Introduction People built different Knowledge Graphs, such as Freebase (Bollacker et al., 2008), to store complex structured information and knowledge about ab- stract and real world. The facts in KG are usually represented in the form of triplets, e.g., (New York, isCityOf, USA). KGs have been widely applied in various intelligent systems such as question an- swering (Yasunaga et al., 2021; Chen et al., 2022), recommender system(Guo et al., 2020; Wong et al., 2021; Zhang et al., 2021) and zero-shot learn- ing(Geng et al., 2022; Chen et al., 2021b). ∗ Corresponding Author. 1Source code of Ruleformer is available at https://github.com/zjukg/ruleformer. isCityOf belongsTo What’s his/her nationality? ambassador (major, diplomacy) (bornIn, Yew York) Rule1. nationality(X, Y) ← livesIn(X, Z) ^ isCityOf(Z, Y) Rule2. nationality(X, Y) ← worksAt(X, Z) ^ belongsTo(Z, Y) localEmployee (bornIn, London) (graduated, Cambridge) Figure 1: Rules in different context. Although these KGs already contain a large num- ber of relations and entities, they still suffer from the incompleteness of facts, whether constructed automatically or manually. In order to further ex- pand KGs, many reasoning methods have been proposed to automated fact exploration, such as knowledge graph embedding (KGE) (Bordes et al., 2013; Trouillon et al., 2016; Dettmers et al., 2018; Sun et al., 2019; Zhang et al., 2019b), graph neural networks (Schlichtkrull et al., 2018), and rule min- ing methods (Ortona et al., 2018; Galárraga et al., 2013; Yang et al., 2017; Sadeghian et al., 2019; Zhang et al., 2019a). Compared with …

Content truncated to 500 words