TranSHER: Translating Knowledge Graph Embedding with Hyper-Ellipsoidal Restriction

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8517 - 8528 December 7-11, 2022 ©2022 Association for Computational Linguistics TranSHER: Translating Knowledge Graph Embedding with Hyper-Ellipsoidal Restriction Yizhi Li1∗, Wei Fan2 , Chao Liu3, Chenghua Lin1†, Jiang Qian3 1 Department of Computer Science, The University of Sheffield, UK 2 Department of Computer Science, University of Central Florida, USA 3 Pingan Technology, China {yizhi.li, c.lin}@sheffield.ac.uk, weifan@knights.ucf.edu, lliuchao666@mail.ustc.edu.cn, jqian104@126.com Abstract Knowledge graph embedding methods are im- portant for the knowledge graph completion (or link prediction) task. One existing efficient method, PairRE, leverages two separate vectors to model complex relations (i.e., 1-to-N, N-to- 1, and N-to-N) in knowledge graphs. How- ever, such a method strictly restricts entities on the hyper-ellipsoid surfaces which limits the optimization of entity distribution, leading to suboptimal performance of knowledge graph completion. To address this issue, we pro- pose a novel score function TranSHER, which leverages relation-specific translations between head and tail entities to relax the constraint of hyper-ellipsoid restrictions. By introduc- ing an intuitive and simple relation-specific translation, TranSHER can provide more di- rect guidance on optimization and capture more semantic characteristics of entities with com- plex relations. Experimental results show that TranSHER achieves significant performance improvements on link prediction and gener- alizes well to datasets in different domains and scales. Our codes are public available at https://github.com/yizhilll/TranSHER. 1 Introduction Knowledge graph is proposed to structurally store human knowledge when the development of com- puter science has brought exponentially growing digitized information. Knowledge graphs have been widely adopted in important applications, such as semantic parsing (Berant et al., 2013), ques- tion generation and answering (Lin et al., 2015a; Hao et al., 2017; Peng et al., 2021), and informa- tion retrieval (Xiong et al., 2017). However, due to the expanding nature and difficulties of construc- tion, knowledge graphs often suffer from incom- pleteness. Thus, knowledge graph completion (or ∗ Part of the work is done at Pingan Technology. † Corresponding author. Optimized Entities (a) PairRE Optimized Entities (b) TranSHER Figure 1: A Comparison Example of Distribution Optimization Between PairRE and TranSHER in 2- dimensional Space. link prediction) becomes a task of great importance from both a research and application perspective. To construct a knowledge graph, entities and re- lations extracted from facts are treated as nodes and edges. A single fact in this graph is represented as a directed relation-specific link between two entities, which is denoted as a triplet like (head, relation, tail). Since knowledge graphs usually contain a large number of entities and relations, knowledge graph embeddings are proposed to efficiently learn the representation of graphs and accomplish the link prediction task with score functions. Intuitions behind the design of score functions can be summa- rized as two important principles: 1) logical reason- ing relation patterns such as symmetry (antisym- metry), inversion, and composition; 2) statistically categorized relation types like 1-to-1, 1-to-N, N- to-1, and N-to-N, where the latter three are called complex relations. Inspired by word2vec (Mikolov et al., 2013), TransE …

Content truncated to 500 words