To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6335–6347 July 9-14, 2023 ©2023 Association for Computational Linguistics To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion Rui Li1∗, Xu Chen2†, Chaozhuo Li3†, Yanming Shen1, Jianan Zhao4, Yujing Wang5, Weihao Han5, Hao Sun5, Weiwei Deng5, Qi Zhang5, Xing Xie3 1Dalian University of Technology, 2Renmin University of China, 3Microsoft Research Asia, 4Université de Montréal, 5Microsoft xu.chen@ruc.edu.cn, cli@microsoft.com Abstract Embedding models have shown great power in knowledge graph completion (KGC) task. By learning structural constraints for each train- ing triple, these methods implicitly memorize intrinsic relation rules to infer missing links. However, this paper points out that the multi- hop relation rules are hard to be reliably mem- orized due to the inherent deficiencies of such implicit memorization strategy, making em- bedding models underperform in predicting links between distant entity pairs. To allevi- ate this problem, we present Vertical Learn- ing Paradigm (VLP), which extends embed- ding models by allowing to explicitly copy tar- get information from related factual triples for more accurate prediction. Rather than solely relying on the implicit memory, VLP directly provides additional cues to improve the gen- eralization ability of embedding models, espe- cially making the distant link prediction sig- nificantly easier. Moreover, we also propose a novel relative distance based negative sam- pling technique (ReD) for more effective op- timization. Experiments demonstrate the va- lidity and generality of our proposals on two standard benchmarks. Our code is available at https://github.com/rui9812/VLP. 1 Introduction Knowledge graphs (KGs) structurally represent hu- man knowledge as a collection of factual triples. Each triple (h, r, t) represents that there is a rela- tion r between head entity h and tail entity t. With the massive human knowledge, KGs facilitate a myriad of downstream applications (Xiong et al., 2017). However, real-world KGs such as Freebase (Bollacker et al., 2008) are far from complete (Bor- des et al., 2013). This motivates substantial re- search on the knowledge graph completion (KGC) task, i.e., automatically inferring missing triples. * Work done during Rui Li’s internship at MSRA. † Correspondence to: Xu Chen <xu.chen@ruc.edu.cn>, Chaozhuo Li <cli@microsoft.com>. Training Memory Test Horizontal Learning Paradigm Figure 1: Learning paradigm of embedding models. As an effective solution for KGC, embedding model learns representations of entities and rela- tions with pre-designed relation operations. For example, TransE (Bordes et al., 2013) represents relations as translations between head and tail en- tities. RESCAL (Nickel et al., 2011), DistMult (Yang et al., 2015) and ComplEx (Trouillon et al., 2016) model the three-way interactions in each triple. RotatE (Sun et al., 2019), QuatE (Zhang et al., 2019) and DualE (Cao et al., 2021) repre- sent relations as rotations in different dimensions. Rot-Pro (Song et al., 2021) further introduces the orthogonal projection for each relation. Essentially, embedding models learn structural constraints for every factual triple during the train- ing period. For example, for each training triple (h, r, t), TransE constrains that the head …

Content truncated to 500 words