Graph Hawkes Transformer for Extrapolated Reasoning on Temporal Knowledge Graphs

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7481 - 7493 December 7-11, 2022 ©2022 Association for Computational Linguistics Graph Hawkes Transformer for Extrapolated Reasoning on Temporal Knowledge Graphs Haohai Sun1∗ Shangyi Geng1∗ Jialun Zhong1 Han Hu2 Kun He1† 1 School of Computer Science and Technology, Huazhong University of Science and Technology 2 Microsoft Research Asia {haohais, shangyigeng, zhongjl}@hust.edu.cn hanhu@microsoft.com brooklet60@hust.edu.cn Abstract Temporal Knowledge Graph (TKG) reasoning has attracted increasing attention due to its enor- mous potential value, and the critical issue is how to model the complex temporal structure information effectively. Recent studies use the method of encoding graph snapshots into hid- den vector space and then performing heuristic deductions, which perform well on the task of entity prediction. However, these approaches cannot predict when an event will occur, and have the following limitations: 1) there are many facts not related to the query that can confuse the model; 2) there exists information forgetting caused by long-term evolutionary processes. To this end, we propose a Graph Hawkes Transformer (GHT) for both TKG en- tity prediction and time prediction tasks in the future time. In GHT, there are two variants of Transformer, which capture the instantaneous structural information and temporal evolution information, respectively, and a new relational continuous-time encoding function to facili- tate feature evolution with the Hawkes pro- cess. Extensive experiments on four public datasets demonstrate its superior performance, especially on long-term evolutionary tasks. 1 Introduction Knowledge Graph (KG), a multi-relational di- rected graph database that stores human knowl- edge and facts, is widely used in downstream ap- plications such as recommendation systems (Guo et al., 2020; Wang et al., 2018), web search (Paul- heim, 2017) and question answering (Saxena et al., 2021). Conventionally, KGs store each fact in the form of a triplet (𝐴���𝐴���𝐴��� 𝐴���𝐴���𝐴���𝐴���, 𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���, 𝐴���𝐴��� 𝐴���𝐴���𝐴���𝐴���). However, many facts may change over time and may contain event-based interaction data. To encode the temporal information, Temporal Knowledge Graph (TKG) is proposed so that *Equal Contribution. †Corresponding author. each temporal fact is stored as a quadruple (𝐴���𝐴���𝐴��� 𝐴���𝐴���𝐴���𝐴���, 𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���, 𝐴���𝐴��� 𝐴���𝐴���𝐴���𝐴���, 𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���𝐴���). Typ- ically, we represent TKG as a sequence of static KG snapshots associated with timestamps. There are two types of reasoning for TKG. In- terpolated reasoning is designed to complete the missing facts on known historical snapshots, while extrapolated reasoning predicts future events or facts. This work focuses on extrapolated reason- ing, a problematic but meaningful task, which can be used for crisis warning, user behavior predic- tion, supply chain management, etc. While most existing works (Han et al., 2021a; Sun et al., 2021) perform link predictions on snapshots at the next timestamp, we consider predicting events that may occur over a long period in the future, which are more appropriate in the real-world setting. The task requires the assistance of known histor- ical facts. A key feature required by the model is to retrieve key information from complex tempo- ral structured data that can help answer the query and make …

Content truncated to 500 words