Query Structure Modeling for Inductive Logical Reasoning Over Knowledge Graphs

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4706–4718 July 9-14, 2023 ©2023 Association for Computational Linguistics Query Structure Modeling for Inductive Logical Reasoning Over Knowledge Graphs Siyuan Wang1, Zhongyu Wei1,2∗, Meng Han3, Zhihao Fan1, Haijun Shan4, Qi Zhang5, Xuanjing Huang5 1School of Data Science, Fudan University, China 2Research Institute of Intelligent and Complex Systems, Fudan University, China 3Huawei Poisson Lab, China 4CEC GienTech Technology Co., Ltd, China 5School of Computer Science, Fudan University, China {wangsy18,zywei,fanzh18,qz,xjhuang}@fudan.edu.cn hanmeng12@huawei.com; haijun.shan@gientech.com Abstract Logical reasoning over incomplete knowledge graphs to answer complex logical queries is a challenging task. With the emergence of new entities and relations in constantly evolv- ing KGs, inductive logical reasoning over KGs has become a crucial problem. However, pre- vious PLMs-based methods struggle to model the logical structures of complex queries, which limits their ability to generalize within the same structure. In this paper, we propose a structure- modeled textual encoding framework for induc- tive logical reasoning over KGs. It encodes linearized query structures and entities using pre-trained language models to find answers. For structure modeling of complex queries, we design stepwise instructions that implic- itly prompt PLMs on the execution order of geometric operations in each query. We fur- ther separately model different geometric oper- ations (i.e., projection, intersection, and union) on the representation space using a pre-trained encoder with additional attention and maxout layers to enhance structured modeling. We con- duct experiments on two inductive logical rea- soning datasets and three transductive datasets. The results demonstrate the effectiveness of our method on logical reasoning over KGs in both inductive and transductive settings. 1 1 Introduction Logical reasoning over knowledge graphs (KGs) aims to answer complex logical queries given large- scale KGs (Guu et al., 2015; Hamilton et al., 2018). Recent years have witnessed increasing attention on logical reasoning over widely used KGs such as Freebase (Bollacker et al., 2008), Yago (Suchanek ∗Corresponding author 1Codes are publicly available at https://github.com/ WangsyGit/InductiveLR. 𝑞!= 𝑣?.∃v:Win(TuringAward, v) ∧ Citizen(Canada, v) ∧ Graduate(v, 𝑣?) Canada Turing Award Win Citizen v 𝑣? Graduate 𝑞#= 𝑣?.∃v :Win(AcademyAwards, v) ∧ Citizen(Canada, v) ∧ MaidenWork(v, 𝑣?) Canada Academy Awards 𝑣? Maiden work (a) Logical query for Training: (b) Logical query for Testing: Win Citizen v 𝑞$= 𝑣?.∃ 𝑣!, 𝑣$:Country(WinterOlympics, 𝑣!) ∧ President(𝑣!, 𝑣$) ∧ Spouse(𝑣$, 𝑣?) Winter Olympics Hold "𝑣" 𝑣? President "𝑣# Spouse 𝑞%= 𝑣?.∃ 𝑣!, 𝑣$ :Director(FireDownBelow, 𝑣!) ∧ Children(𝑣!, 𝑣$) ∧ Profession(𝑣$, 𝑣?) Fire Down Below Director "𝑣" 𝑣? Children "𝑣# Profession ”Where did Canadian citizens with Turing Award graduate?” ”Who are spouses of presidents of countries that held Winter Olympics?” ”What are the maiden works of Canadian Academy Awards winners?” ”What are the professions of the children of Fire Down Below’s director?” Figure 1: Examples of inductive logical reasoning over KGs: testing queries contain unseen entities and rela- tions (in red) during training. Each query is associated with an intrinsic logical structure and its natural lan- guage interpretation. et al., 2007), NELL (Carlson et …

Content truncated to 500 words