Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2713–2722 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2713 Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding Yun Tang, Jing Huang, Guangtao Wang, Xiaodong He, Bowen Zhou JD AI Research {yun.tang,jing.huang,guangtao.wang,xiaodong.he,bowen.zhou}@jd.com Abstract Distance-based knowledge graph embeddings have shown substantial improvement on the knowledge graph link prediction task, from TransE to the latest state-of-the-art RotatE. However, complex relations such as N-to-1, 1-to-N and N-to-N still remain challenging to predict. In this work, we propose a novel distance-based approach for knowledge graph link prediction. First we extend the RotatE from 2D complex domain to high dimensional space with orthogonal transforms to model relations. The orthogonal transform embed- ding for relations keeps the capability for modeling symmetric/anti-symmetric, inverse and compositional relations while achieves better modeling capacity. Second, the graph context is integrated into distance scoring functions directly. Specifically, graph context is explicitly modeled via two directed context representations. Each node embedding in knowledge graph is augmented with two context representations, which are computed from the neighboring outgoing and incoming nodes/edges respectively. The proposed approach improves prediction accuracy on the difficult N-to-1, 1-to-N and N-to-N cases. Our experimental results show that it achieves state-of-the-art results on two common bench- marks FB15k-237 and WNRR-18, especially on FB15k-237 which has many high in-degree nodes. Code available at https://github. com/JD-AI-Research-Silicon-Valley/ KGEmbedding-OTE. 1 Introduction Knowledge graph is a multi-relational graph whose nodes represent entities and edges denote relation- ships between entities. Knowledge graphs store facts about people, places and world from various sources. Those facts are kept as triples (head en- tity, relation, tail entity) and denoted as (h, r, t). A large number of knowledge graphs, such as Freebase (Bollacker et al., 2008), DBpedia (Auer et al., 2007), NELL (Carlson et al., 2010) and YAGO3 (Mahdisoltani et al., 2013), have been built over the years and successfully applied to many domains such as recommendation and question an- swering (Bordes et al., 2014; Zhang et al., 2016). However, these knowledge graphs need to be up- dated with new facts periodically. Therefore many knowledge graph embedding methods have been proposed for link prediction that is used for knowl- edge graph completion. Knowledge graph embedding represents enti- ties and relations in continuous vector spaces. Started from a simple and effective approach called TransE (Bordes et al., 2013), many knowledge graph embedding methods have been proposed, such as TransH (Wang et al., 2014), DistMult (Yang et al., 2014), ConvE (Dettmers et al., 2018) to the latest RotatE (Sun et al., 2019) and QuatE (Zhang et al., 2019). Though much progress has been made, 1-to- N, N-to-1, and N-to-N relation predictions (Bor- des et al., 2013; Wang et al., 2014) still remain challenging. In Figure 1, relation “profession” demonstrates an N-to-N example and the corre- sponding edges are highlighted as green. Assum- ing the triple (SergeiRachmaninoff, Profession, Pianist) is unknown. The link prediction model takes “SergeiRachmaninoff” and relation “Profes- sion” and rank all …

Content truncated to 500 words