SQUIRE: A Sequence-to-sequence Framework for Multi-hop Knowledge Graph Reasoning

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1649 - 1662 December 7-11, 2022 ©2022 Association for Computational Linguistics SQUIRE: A Sequence-to-sequence Framework for Multi-hop Knowledge Graph Reasoning Yushi Bai1,2, Xin Lv1,2, Juanzi Li1,2, Lei Hou1,2∗, Yincen Qu3, Zelin Dai3, Feiyu Xiong3 1Department of Computer Science and Technology, BNRist; 2KIRC, Institute for Artificial Intelligence; Tsinghua University, Beijing 100084, China 3Alibaba Group, Hangzhou, China {bys22@mails., lijuanzi@, houlei@}tsinghua.edu.cn Abstract Multi-hop knowledge graph (KG) reasoning has been widely studied in recent years to pro- vide interpretable predictions on missing links with evidential paths. Most previous works use reinforcement learning (RL) based meth- ods that learn to navigate the path towards the target entity. However, these methods suf- fer from slow and poor convergence, and they may fail to infer a certain path when there is a missing edge along the path. Here we present SQUIRE, the first Sequence-to-sequence based multi-hop reasoning framework, which utilizes an encoder-decoder Transformer structure to translate the query to a path. Our framework brings about two benefits: (1) It can learn and predict in an end-to-end fashion, which gives better and faster convergence; (2) Our trans- former model does not rely on existing edges to generate the path, and has the flexibility to complete missing edges along the path, espe- cially in sparse KGs. Experiments on standard and sparse KGs show that our approach yields significant improvement over prior methods, while converging 4x-7x faster. 1 Introduction Knowledge graph (KG) provides structural knowl- edge about entities and relations in real world in the form of triples. Each edge in the graph, con- necting two entities with a relation, represents a triple fact (h, r, t). Knowledge graph supports a va- riety of downstream tasks, such as question answer- ing (Hao et al., 2017), information retrieval (Xiong et al., 2017a) and hierarchical reasoning (Bai et al., 2021). However, practical KGs often suffer from in- completeness, thus proposing the task of KG com- pletion, such as predicting the tail entity t given (h, r). A popular approach for such a challenge is knowledge graph embedding (KGE) (Bordes et al., 2013; Dettmers et al., 2018), which infers a missing edge in a complete black-box manner. ∗ Corresponding Author Albert Einstein Mileva Marić German Hans Albert Einstein Germany spouse born in child language child? native language? ETH Zurich educated at Figure 1: An example of multi-hop reasoning in an in- complete knowledge graph. The missing links (dashed arrows) can be inferred from existing links (solid ar- rows). However, such an evidential path may not be inferred when there is a missing edge (gray dashed ar- row) along the path. To strengthen the interpretability of KG comple- tion, (Das et al., 2018) proposes multi-hop knowl- edge graph reasoning. Given a triple query (h, r), the task aims not only to predict the tail entity t, but to give the evidential path from h to t that indicates the inference process, e.g., we can infer (Albert, native language, ?) from the …

Content truncated to 500 words