Knowledge Graph Representation Learning using Ordinary Differential Equations

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9529–9548 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9529 Knowledge Graph Representation Learning using Ordinary Differential Equations Mojtaba Nayyeri University Of Bonn nayyeri@iai.uni-bonn.de Chengjin Xu University Of Bonn xuc@iai.uni-bonn.de Franca Hoffmann University Of Bonn franca.hoffmann@hcm.uni-bonn.de Mirza Mohtashim Alam Nature-Inspired Machine Intelligence-InfAI mohtasim@infai.org Jens Lehmann Fraunhofer IAIS jens.lehmann@iais.fraunhofer.de Sahar Vahdati Nature-Inspired Machine Intelligence-InfAI vahdati@infai.org Abstract Knowledge Graph Embeddings (KGEs) have shown promising performance on link predic- tion tasks by mapping the entities and rela- tions from a knowledge graph into a geomet- ric space. The capability of KGEs in pre- serving graph characteristics including struc- tural aspects and semantics, highly depends on the design of their score function, as well as the inherited abilities from the underlying geometry. Many KGEs use the Euclidean ge- ometry which renders them incapable of pre- serving complex structures and consequently causes wrong inferences by the models. To address this problem, we propose a neuro dif- ferential KGE that embeds nodes of a KG on the trajectories of Ordinary Differential Equa- tions (ODEs). To this end, we represent each relation (edge) in a KG as a vector field on sev- eral manifolds. We specifically parameterize ODEs by a neural network to represent com- plex manifolds and complex vector fields on the manifolds. Therefore, the underlying em- bedding space is capable to assume the shape of various geometric forms to encode hetero- geneous subgraphs. Experiments on synthetic and benchmark datasets using state-of-the-art KGE models justify the ODE trajectories as a means to enable structure preservation and consequently avoiding wrong inferences. 1 Introduction Knowledge Graphs (KGs) have a significant im- pact on machine learning approaches (Wang et al., 2017). A KG usually represents factual knowledge in triples of the form (entity, relation, entity) e.g., (Plato, influences, Kant). The nodes of a KG rep- resent entities and the links denote the relations. Although quantitatively KGs are often large-scale with millions of triples, they are usually incom- plete i.e. do not capture all knowledge within a do- main of interest. To address this problem, various approaches have been used so far, among which link prediction using KG embeddings (KGE) at- tracted growing attention. KGE models map enti- ties (e) and relations (r) of a KG from a symbolic domain to a geometric space (e.g. a vector space). Such embedding models employ a score function to perform the link prediction task, which uses the learned embedding vectors (et, r, et+1) of a triple (et, r, et+1) to compute its plausibility. This allows to rank triples by their score, where a correct triple should obtain a higher score and a lower rank than an incorrect triple. KGE models have been predom- inantly studied with a focus on the triple level. However, in a broader view, triples of a graph form specific subgraphs (i.e. structural patterns distributed in the graph) in local structures. The preservation of subgraphs in the learned represen- tations is a major challenge. In this regard, …

Content truncated to 500 words