TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5730–5746, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 5730 TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion Jiapeng Wu Meng Cao Jackie Chi Kit Cheung William L. Hamilton School of Computer Science, McGill University, Montreal, QC, Canada MILA, Montreal, QC, Canada {jiapeng.wu@mail, meng.cao@mail, jcheung@cs, wlh@cs}.mcgill.ca Abstract Inferring missing facts in temporal knowledge graphs (TKGs) is a fundamental and challeng- ing task. Previous works have approached this problem by augmenting methods for static knowledge graphs to leverage time-dependent representations. However, these methods do not explicitly leverage multi-hop structural in- formation and temporal facts from recent time steps to enhance their predictions. Addition- ally, prior work does not explicitly address the temporal sparsity and variability of entity dis- tributions in TKGs. We propose the Temporal Message Passing (TeMP) framework to ad- dress these challenges by combining graph neural networks, temporal dynamics models, data imputation and frequency-based gating techniques. Experiments1 on standard TKG tasks show that our approach provides substan- tial gains compared to the previous state of the art, achieving a 10.7% average relative im- provement in Hits@10 across three standard benchmarks. Our analysis also reveals im- portant sources of variability both within and across TKG datasets, and we introduce several simple but strong baselines that outperform the prior state of the art in certain settings. 1 Introduction The ability to infer missing facts in temporal knowledge graphs is essential for applications such as event prediction (Leblay and Chekol, 2018; De Winter et al., 2018), question answering (Jia et al., 2018), social network analysis (Zhou et al., 2018; Trivedi et al., 2019) and recommendation systems (Kumar et al., 2018). Whereas static knowledge graphs (KGs) repre- sent facts as triples (e.g., (Obama, visit, China)), temporal knowledge graphs (TKGs) addition- ally associate each triple with a timestamp (e.g., 1Code and data are published at https://github. com/JiapengWu/TeMP (Obama, visit, China, 2014)). Figure 1 shows a subgraph of such TKG. Usually, TKGs are as- sumed to consist of discrete timestamps (Jiang et al., 2016), meaning that they can be represented as a sequence of static KG snapshots, and the task of inferring missing facts across these snapshots is referred to as temporal knowledge graph complete- tion (TKGC). Recent works on TKGC have largely focused on developing time-dependent scoring functions, which score the likelihood of missing facts and build closely upon popular representation learn- ing methods for static KGs (Dasgupta et al., 2018; Jiang et al., 2016; Goel et al., 2019; Xu et al., 2019; Lacroix et al., 2020). However, while powerful, these existing methods do not properly account for multi-hop structural information in TKGs, and they lack the ability to explicitly leverage temporal facts in nearby KG snapshots to answer queries. Know- ing facts like (Obama, make agreement with, China, 2013) or (Obama, visit, China, 2012) is useful for answering the query (Obama, visit, ?, 2014). Moreover—and perhaps more importantly— there are also serious challenges regarding tem- poral variability and temporal …

Content truncated to 500 words