Learning Neural Ordinary Equations for Forecasting Future Links on Temporal Knowledge Graphs

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8352–8364 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8352 Learning Neural Ordinary Equations for Forecasting Future Links on Temporal Knowledge Graphs Zhen Han∗1,2, Zifeng Ding∗1,2, Yunpu Ma∗1, Yujia Gu3 Volker Tresp†1,2 1Institute of Informatics, LMU Munich 2 Corporate Technology, Siemens AG 3Department of Electrical and Computer Engineering, Technical University of Munich zhen.han@campus.lmu.de, cognitive.yunpu@gmail.com {zifeng.ding, volker.tresp}@siemens.com, yujia.gu@tum.de Abstract There has been an increasing interest in in- ferring future links on temporal knowledge graphs (KG). While links on temporal KGs vary continuously over time, the existing ap- proaches model the temporal KGs in discrete state spaces. To this end, we propose a novel continuum model by extending the idea of neu- ral ordinary differential equations (ODEs) to multi-relational graph convolutional networks. The proposed model preserves the continuous nature of dynamic multi-relational graph data and encodes both temporal and structural in- formation into continuous-time dynamic em- beddings. In addition, a novel graph transition layer is applied to capture the transitions on the dynamic graph, i.e., edge formation and disso- lution. We perform extensive experiments on five benchmark datasets for temporal KG rea- soning, showing our model’s superior perfor- mance on the future link forecasting task. 1 Introduction Reasoning on relational data has long been consid- ered an essential subject in artificial intelligence with wide applications, including decision sup- port and question answering. Recently, reasoning on knowledge graphs has gained increasing inter- est (Ren and Leskovec, 2020; Das et al., 2018). A Knowledge Graph (KG) is a graph-structured knowledge base to store factual information. KGs represent facts in the form of triples (s, r, o), e.g., (Bob, livesIn, New York), in which s (subject) and o (object) denote nodes (entities), and r denotes the edge type (relation) between s and o. Knowl- edge graphs are commonly static and store facts in their current state. In reality, however, the relations between entities often change over time. For exam- ple, if Bob moves to California, the triple of (Bob, livesIn, New York) will be invalid. To this end, temporal knowledge graphs (tKG) were introduced. ∗Equal contribution. †Corresponding author. A tKG represents a temporal fact as a quadruple (s, r, o, t) by extending a static triple with time t, describing that this fact is valid at time t. In recent years, several sizable temporal knowledge graphs, such as ICEWS (Boschee et al., 2015), have been developed that provide widespread availability of such data and enable reasoning on temporal KGs. While lots of work (García-Durán et al., 2018; Goel et al., 2020; Lacroix et al., 2020) focus on the tem- poral KG completion task and predict missing links at observed timestamps, recent work (Jin et al., 2019; Trivedi et al., 2017) paid attention to forecast future links of temporal KGs. In this work, we focus on the temporal KG forecasting task, which is more challenging than the completion task. Most existing work (Jin et al., 2019; Zhu et al., 2020) models temporal KGs …

Content truncated to 500 words