DyERNIE: Dynamic Evolution of Riemannian Manifold Embeddings for Temporal Knowledge Graph Completion

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7301–7316, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 7301 DyERNIE: Dynamic Evolution of Riemannian Manifold Embeddings for Temporal Knowledge Graph Completion Zhen Han1,2, Peng Chen2,3, Yunpu Ma1,2˚, Volker Tresp1,2˚ 1Institute of Informatics, LMU Munich 2 Corporate Technology, Siemens AG 3Department of Informatics, Technical University of Munich zhen.han@campus.lmu.de, peng.chen@tum.de cognitive.yunpu@gmail.com, volker.tresp@siemens.com Abstract There has recently been increasing interest in learning representations of temporal knowl- edge graphs (KGs), which record the dynamic relationships between entities over time. Tem- poral KGs often exhibit multiple simultane- ous non-Euclidean structures, such as hierar- chical and cyclic structures. However, exist- ing embedding approaches for temporal KGs typically learn entity representations and their dynamic evolution in the Euclidean space, which might not capture such intrinsic struc- tures very well. To this end, we propose Dy- ERNIE, a non-Euclidean embedding approach that learns evolving entity representations in a product of Riemannian manifolds, where the composed spaces are estimated from the sec- tional curvatures of underlying data. Product manifolds enable our approach to better re- flect a wide variety of geometric structures on temporal KGs. Besides, to capture the evo- lutionary dynamics of temporal KGs, we let the entity representations evolve according to a velocity vector defined in the tangent space at each timestamp. We analyze in detail the contribution of geometric spaces to represen- tation learning of temporal KGs and evaluate our model on temporal knowledge graph com- pletion tasks. Extensive experiments on three real-world datasets demonstrate significantly improved performance, indicating that the dy- namics of multi-relational graph data can be more properly modeled by the evolution of em- beddings on Riemannian manifolds. 1 Introduction Learning from relational data has long been con- sidered as a key challenge in artificial intelligence. In recent years, several sizable knowledge graphs (KGs), e.g. Freebase (Bollacker et al., 2008) and Wikidata (Vrandeˇci´c and Krötzsch, 2014), have ˚Corresponding author. been developed that provide widespread availabil- ity of such data and enabled improvements to a plethora of downstream applications such as rec- ommender systems (Hildebrandt et al., 2019) and question answering (Zhang et al., 2018). KGs are multi-relational, directed graphs with labeled edges, where each edge corresponds to a fact and can be represented as a triple, such as (John, lives in, Van- couver). Common knowledge graphs are static and store facts at their current state. In reality, however, multi-relational data are often time-dependent. For example, the political relationship between two countries might intensify because of trade fights. Thus, temporal knowledge graphs were introduced, such as ICEWS (Boschee et al., 2015) and GDELT (Leetaru and Schrodt, 2013), that capture temporal aspects of facts in addition to their multi-relational nature. In these datasets, temporal facts are repre- sented as a quadruple by extending the static triplet with a timestamp describing when these facts oc- curred, i.e. (Barack Obama, inaugurated, as presi- dent of the US, 2009). Since real-world temporal KGs are usually incomplete, the task of link predic- tion on temporal …

Content truncated to 500 words