RelWalk - A Latent Variable Model Approach to Knowledge Graph Embedding

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1551–1565 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1551 RelWalk – A Latent Variable Model Approach to Knowledge Graph Embedding Danushka Bollegala1∗, Huda Hakami2, Yuichi Yoshida3 and Ken-ichi Kawarabayashi3 1University of Liverpool, Amazon. 2Taif University, Saudi Arabia 3National Institute of Informatics, Japan danushka@liverpool.ac.uk, hahakami@tu.edu.sa {yyoshida,k keniti}@nii.ac.jp Abstract Embedding entities and relations of a knowl- edge graph in a low-dimensional space has shown impressive performance in predicting missing links between entities. Although pro- gresses have been achieved, existing methods are heuristically motivated and theoretical un- derstanding of such embeddings is compara- tively underdeveloped. This paper extends the random walk model (Arora et al., 2016a) of word embeddings to Knowledge Graph Em- beddings (KGEs) to derive a scoring function that evaluates the strength of a relation R be- tween two entities h (head) and t (tail). More- over, we show that marginal loss minimisa- tion, a popular objective used in much prior work in KGE, follows naturally from the log- likelihood ratio maximisation under the prob- abilities estimated from the KGEs according to our theoretical relationship. We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowl- edge graph. Using the derived objective, ac- curate KGEs are learnt from FB15K237 and WN18RR benchmark datasets, providing em- pirical evidence in support of the theory. 1 Introduction Knowedge graphs (KGs) such as Freebase (Bol- lacker et al., 2008) organise information in the form of graphs, where entities are represented by the vertices and the relations between two entities are represented by the edges that connect the corre- sponding vertices. Despite the best efforts to create complete and large-scale KGs, most KGs remain incomplete and do not represent all the relations that exist between entities (Min et al., 2013). In particular, new entities are constantly being gener- ated, and new relations are formed between new as ∗Danushka Bollegala holds concurrent appointments as a Professor at University of Liverpool and as an Amazon Scholar. This paper describes work performed at the Univer- sity of Liverpool and is not associated with Amazon. well as existing entities. Therefore, it is unrealistic to assume that a real-world KG would be complete at any given time point. Developing approaches for KG completion is an important research field associated with KGs. KG components can be embedded into numeri- cal formats by learning representations (a.k.a em- beddings) for the entities and relations in a given KG. The learnt KGEs can be used for link pre- diction, which is the task of predicting whether a particular relation exists between two given entities in the KG. Specifically, given KGEs for entities and relations, in link prediction, we predict R that is most likely to exist between h and t according to some scoring formula. Thus, by embedding entities and relations that exist in a KG in some (possibly lower-dimensional and latent) space, we can in- fer previously unseen relations …

Content truncated to 500 words