Knowledge Graph Embedding by Adaptive Limit Scoring Loss Using Dynamic Weighting Strategy

Findings of the Association for Computational Linguistics: ACL 2022, pages 1153 - 1163 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Knowledge Graph Embedding by Adaptive Limit Scoring Loss Using Dynamic Weighting Strategy Jinfa Yang, Xianghua Ying*, Yongjie Shi, Xin Tong, Ruibin Wang, Taiyan Chen, Bowei Xing Key Laboratory of Machine Perception (MOE) School of Artificial Intelligence, Peking University {jinfayang, xhying, shiyongjie, xin_tong, robin_wang}@pku.edu.cn, chenty@stu.pku.edu.cn, 2017xbw@pku.edu.cn Abstract Knowledge graph embedding aims to represent entities and relations as low-dimensional vec- tors, which is an effective way for predicting missing links in knowledge graphs. Design- ing a strong and effective loss framework is essential for knowledge graph embedding mod- els to distinguish between correct and incor- rect triplets. The classic margin-based ranking loss limits the scores of positive and negative triplets to have a suitable margin. The recently proposed Limit-based Scoring Loss indepen- dently limits the range of positive and negative triplet scores. However, these loss frameworks use equal or fixed penalty terms to reduce the scores of positive and negative sample pairs, which is inflexible in optimization. Our intu- ition is that if a triplet score deviates far from the optimum, it should be emphasized. To this end, we propose Adaptive Limit Scoring Loss, which simply re-weights each triplet to high- light the less-optimized triplet scores. We apply this loss framework to several knowledge graph embedding models such as TransE, TransH and ComplEx. The experimental results on link pre- diction and triplet classification show that our proposed method has achieved performance on par with the state of the art. 1 Introduction Knowledge graphs are usually collections of fac- tual triplets — (head entity, relation, tail entity), also known as (subject, predicate, object), which represent human knowledge of the real world in a structured way. There are some outstanding knowl- edge graphs, such as WordNet (Miller, 1995), Free- base (Bollacker et al., 2008), DBpedia (Lehmann et al., 2015), YAGO (Suchanek et al., 2007). They have gained widespread attention for their success- ful usage in various applications, e.g., question answering (Bordes et al., 2014; Huang et al., 2019), *Corresponding Author   0 p S n S   0 (a) (b) T T T T p p  n n  p S n S 1P 2P 3P 1P 2P 3P Figure 1: Comparison between the popular optimization manner of reducing (𝐴���𝐴���, 𝐴���𝐴���) and the proposed reduc- ing (𝛹���𝐴���𝐴���𝐴���, 𝛹���𝐴���𝐴���𝐴���). (a) Reducing (𝐴���𝐴���, 𝐴���𝐴���) is prone to inflexible optimization (𝐴���1, 𝐴���2 and 𝐴���3 all have equal gradients with respect to 𝐴���𝐴��� and 𝐴���𝐴���), as well as poten- tial overlapping problem (both 𝐴��� and 𝐴��� ′ on the decision boundary are acceptable). (b) With (𝛹���𝐴���𝐴���𝐴���, 𝛹���𝐴���𝐴���𝐴���), the 𝐴��� 𝐴𝐴��� dynamically adjusts its gradients on 𝐴���𝐴��� and 𝐴���𝐴���, and thus benefits from a flexible optimization process. For 𝐴���1, it emphasizes on increasing 𝐴���𝐴���; for 𝐴���3, it empha- sizes on reducing 𝐴���𝐴���. Moreover, it aggregates 𝐴��� and 𝐴��� ′ on the circular decision boundary, which can alleviate the overlap problem. recommendation systems (Zhou et al., …

Content truncated to 500 words