PairRE: Knowledge Graph Embeddings via Paired Relation Vectors

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4360–4369 August 1–6, 2021. ©2021 Association for Computational Linguistics 4360 PairRE: Knowledge Graph Embeddings via Paired Relation Vectors Linlin Chao, Jianshan He, Taifeng Wang, Wei Chu AntGroup {chulin.cll,yebai.hjs}@antgroup.com {taifeng.wang,wei.chu}@alibaba-inc.com Abstract Distance based knowledge graph embedding methods show promising results on link pre- diction task, on which two topics have been widely studied: one is the ability to handle complex relations, such as N-to-1, 1-to-N and N-to-N, the other is to encode various rela- tion patterns, such as symmetry/antisymmetry. However, the existing methods fail to solve these two problems at the same time, which leads to unsatisfactory results. To mitigate this problem, we propose PairRE, a model with paired vectors for each relation representation. The paired vectors enable an adaptive adjust- ment of the margin in loss function to fit for complex relations. Besides, PairRE is capable of encoding three important relation patterns, symmetry/antisymmetry, inverse and composi- tion. Given simple constraints on relation rep- resentations, PairRE can encode subrelation further. Experiments on link prediction bench- marks demonstrate the proposed key capabili- ties of PairRE. Moreover, We set a new state- of-the-art on two knowledge graph datasets of the challenging Open Graph Benchmark. 1 Introduction Knowledge graphs store huge amounts of struc- tured data in the form of triples, with projects such as WordNet (Miller, 1995), Freebase (Bollacker et al., 2008), YAGO (Suchanek et al., 2007) and DBpedia (Lehmann et al., 2015). They have gained widespread attraction from their successful use in tasks such as question answering (Bordes et al., 2014), semantic parsing (Berant et al., 2013), and named entity disambiguation (Zheng et al., 2012) and so on. Since most knowledge graphs suffer from incom- pleteness, predicting missing links between entities has been a fundamental problem. This problem is named as link prediction or knowledge graph completion. Knowledge graph embedding meth- ods, which embed all entities and relations into a low dimensional space, have been proposed for this problem. Distance based embedding methods from TransE (Bordes et al., 2013) to the recent state-of-the-art RotatE (Sun et al., 2019) have shown substan- tial improvements on knowledge graph comple- tion task. Two major problems have been widely studied. The first one refers to handling of 1-to- N, N-to-1, and N-to-N complex relations (Bordes et al., 2013; Lin et al., 2015). In case of the 1-to- N relations, given triples like (StevenSpielberg, DirectorOf, ?), distance based models should make all the corresponding entities about film name like Jaws and JurassicPark have closer dis- tance to entity StevenSpielberg after transforma- tion via relation DirectorOf. The difficulty is that all these entities should have different repre- sentations. Same issue happens in cases of N-to-N and N-to-1 relations. The latter is learning and inferring relation patterns according to observed triples, as the success of knowledge graph com- pletion heavily relies on this ability (Bordes et al., 2013; Sun et al., 2019). There are various types …

Content truncated to 500 words