Time-dependent Entity Embedding is not All You Need: A Re-evaluation of Temporal Knowledge Graph Completion Models under a Unified Framework

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8104–8118 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8104 Time-dependent Entity Embedding is not All You Need: A Re-evaluation of Temporal Knowledge Graph Completion Models under a Unified Framework Zhen Han∗1,2, Gengyuan Zhang∗1, Yunpu Ma†1, Volker Tresp†1,2 1Institute of Informatics, LMU Munich 2 Corporate Technology, Siemens AG zhen.han@campus.lmu.de, gengyuanmax@gmail.com cognitive.yunpu@gmail.com, volker.tresp@siemens.com Abstract Various temporal knowledge graph (KG) com- pletion models have been proposed in the re- cent literature. The models usually contain two parts, a temporal embedding layer and a score function derived from existing static KG modeling approaches. Since the approaches differ along several dimensions, including dif- ferent score functions and training strategies, the individual contributions of different tem- poral embedding techniques to model perfor- mance are not always clear. In this work, we systematically study six temporal embedding approaches and empirically quantify their per- formance across a wide range of configura- tions with about 4000 experiments and 19000 GPU hours. We classify the temporal em- beddings into two classes: (1) timestamp em- beddings and (2) time-dependent entity embed- dings. Despite the common belief that the latter is more expressive, an extensive exper- imental study shows that timestamp embed- dings can achieve on-par or even better per- formance with significantly fewer parameters. Moreover, we find that when trained appro- priately, the relative performance differences between various temporal embeddings often shrink and sometimes even reverse when com- pared to prior results. For example, TTransE (Leblay and Chekol, 2018), one of the first temporal KG models, can outperform more recent architectures on ICEWS datasets. To foster further research, we provide the first unified open-source framework for temporal KG completion models with full composabil- ity, where temporal embeddings, score func- tions, loss functions, regularizers, and the ex- plicit modeling of reciprocal relations can be combined arbitrarily. 1 Introduction The Knowledge Graph (KG), a graph-structured knowledge base, has gained increasing interest as ∗Equal contribution. †Corresponding author. a promising way to store factual knowledge. KGs represent facts in the form of triples (s, r, o), e.g., (Bob, livesIn, New York), in which s (subject) and o (object) denote nodes (entities) and r denotes the edge type (relation) between s and o. Knowl- edge graphs are commonly static and store facts in their current state. In reality, however, the rela- tions between entities often change over time. For example, if Bob moves to California, the triple of (Bob, livesIn, New York) will be invalid. To this end, temporal knowledge graphs (tKGs) have been introduced to capture temporal aspects of facts in addition to their multi-relational nature. A tKG represents a temporal fact as a quadruple (s, r, o, t) by extending a static triple with time t, describing that this fact is valid at time t. Figure 2 in the appendix depicts an exemplary temporal KG. To address the inherent incompleteness of temporal KGs, Tresp et al. (2015) proposed the first tKG model. Afterwards, a line of work emerged that extends …

Content truncated to 500 words