Recurrent Event Network: Autoregressive Structure Inferenceover Temporal Knowledge Graphs

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6669–6683, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6669 Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs Woojeong Jin1 Meng Qu2 3 Xisen Jin1 Xiang Ren1 1Department of Computer Science, University of Southern California 2MILA - Quebec AI Institute 3University of Montr´eal {woojeong.jin,xisenjin,xiangren}@usc.edu meng.qu@umontreal.ca Abstract Knowledge graph reasoning is a critical task in natural language processing. The task be- comes more challenging on temporal knowl- edge graphs, where each fact is associated with a timestamp. Most existing methods focus on reasoning at past timestamps and they are not able to predict facts happening in the future. This paper proposes Recurrent Event Network (RE-NET), a novel autoregressive architecture for predicting future interactions. The occur- rence of a fact (event) is modeled as a proba- bility distribution conditioned on temporal se- quences of past knowledge graphs. Specifi- cally, our RE-NET employs a recurrent event encoder to encode past facts, and uses a neigh- borhood aggregator to model the connection of facts at the same timestamp. Future facts can then be inferred in a sequential manner based on the two modules. We evaluate our proposed method via link prediction at future times on five public datasets. Through extensive exper- iments, we demonstrate the strength of RE- NET, especially on multi-step inference over future timestamps, and achieve state-of-the-art performance on all five datasets1. 1 Introduction Knowledge graphs (KGs), which store real-world facts, are vital in various natural language process- ing applications (Bordes et al., 2013; Schlichtkrull et al., 2018; Kazemi et al., 2019). Due to the high cost of annotating facts, most knowledge graphs are far from complete, and thus predicting missing facts (a.k.a., knowledge graph reasoning) becomes an important task. Most existing efforts study rea- soning on standard knowledge graphs, where each fact is represented as a triple of subject entity, ob- ject entity and the relation between them. How- ever, in practice, each fact may not be true forever, 1https://github.com/INK-USC/RE-Net Criticize Threaten Consult Make a statement Time 𝑡" 𝑡# 𝑡$ Threaten Consult Make a statement Protestor (Iran) Criticize Praise ? ? Iran South Korea North Korea Macron Pakistan Pakistan Figure 1: Example temporal knowledge subgraphs. Each edge or interaction between entities is associated with temporal information and a set of interactions build a multi-relational graph at each time. Our task is to predict interactions and build graphs at future times. and hence it is useful to associate each fact with a timestamp as a constraint, yielding a temporal knowledge graph (TKG). Fig. 1 shows example subgraphs of a temporal knowledge graph. Despite the ubiquitousness of TKGs, methods for reasoning over such kind of data are relatively unexplored. Given a temporal knowledge graph with times- tamps varying from t0 to tT , TKG reasoning pri- marily has two settings - interpolation and extrap- olation. In the interpolation setting, new facts are predicted for time t such that t0 ≤ t ≤ tT (Garc´ıa- Dur´an et al., 2018; …

Content truncated to 500 words