TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8306–8319 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8306 TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting Haohai Sun1∗ Jialun Zhong1∗ Yunpu Ma2∗ Zhen Han2,3∗ Kun He1† 1 School of Computer Science and Technology, Huazhong University of Science and Technology 2 Institute of Informatics, LMU Munich 3 Corporate Technology, Siemens AG {haohais, zhongjl}@hust.edu.cn cognitive.yunpu@gmail.com, zhen.han@campus.lmu.de brooklet60@hust.edu.cn Abstract Temporal knowledge graph (TKG) reasoning is a crucial task that has gained increasing re- search interest in recent years. Most exist- ing methods focus on reasoning at past times- tamps to complete the missing facts, and there are only a few works of reasoning on known TKGs to forecast future facts. Compared with the completion task, the forecasting task is more difficult and faces two main challenges: (1) how to effectively model the time informa- tion to handle future timestamps? (2) how to make inductive inference to handle previously unseen entities that emerge over time? To ad- dress these challenges, we propose the first reinforcement learning method for forecast- ing. Specifically, the agent travels on histor- ical knowledge graph snapshots to search for the answer. Our method defines a relative time encoding function to capture the timespan in- formation, and we design a novel time-shaped reward based on Dirichlet distribution to guide the model learning. Furthermore, we propose a novel representation method for unseen en- tities to improve the inductive inference abil- ity of the model. We evaluate our method for this link prediction task at future times- tamps. Extensive experiments on four bench- mark datasets demonstrate substantial perfor- mance improvement meanwhile with higher explainability, less calculation, and fewer pa- rameters when compared with existing state- of-the-art methods. 1 Introduction Storing a wealth of human knowledge and facts, Knowledge Graphs (KGs) are widely used for many downstream Artificial Intelligence (AI) appli- cations, such as recommendation systems (Guo et al., 2020), dialogue generation (Moon et al., 2019), and question answering (Zhang et al., 2018). KGs store facts in the form of triples *Equal Contribution. †Corresponding author. (푒푠, 푟, 푒표), i.e. (subject entity, predicate/relation, object entity), such as (LeBron_James, plays_for, Cleveland_Cavaliers). Each triple corresponds to a labeled edge of a multi-relational directed graph. However, facts constantly change over time. To reflect the timeliness of facts, Tempo- ral Knowledge Graphs (TKGs) additionally asso- ciate each triple with a timestamp (푒푠, 푟, 푒표, 푡), e.g., (LeBron_James, plays_for, Cleveland_Cavaliers, 2014-2018). Usually, we represent a TKG as a sequence of static KG snapshots. TKG reasoning is a process of inferring new facts from known facts, which can be divided into two types, interpolation and extrapolation. Most existing methods (Jiang et al., 2016a; Dasgupta et al., 2018; Goel et al., 2020; Wu et al., 2020) focus on interpolated TKG reasoning to complete the missing facts at past timestamps. In contrast, extrapolated TKG reasoning focuses on forecasting future facts (events). In this work, we focus on extrapolated TKG reasoning by designing a model for the link prediction …

Content truncated to 500 words