HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8095–8107 July 9-14, 2023 ©2023 Association for Computational Linguistics HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level Haoran Luo1, Haihong E1∗, Yuhao Yang2, Yikai Guo3, Mingzhi Sun1, Tianyu Yao1, Zichen Tang1, Kaiyang Wan1, Meina Song1, Wei Lin4 1School of Computer Science, Beijing University of Posts and Telecommunications, China 2School of Automation Science and Electrical Engineering, Beihang University, China 3Beijing Institute of Computer Technology and Application, China 4Inspur Group Co., Ltd., China {luohaoran, ehaihong}@bupt.edu.cn Abstract Link Prediction on Hyper-relational Knowl- edge Graphs (HKG) is a worthwhile endeavor. HKG consists of hyper-relational facts (H- Facts), composed of a main triple and sev- eral auxiliary attribute-value qualifiers, which can effectively represent factually comprehen- sive information. The internal structure of HKG can be represented as a hypergraph- based representation globally and a semantic sequence-based representation locally. How- ever, existing research seldom simultaneously models the graphical and sequential struc- ture of HKGs, limiting HKGs’ representation. To overcome this limitation, we propose a novel Hierarchical Attention model for HKG Embedding (HAHE), including global-level and local-level attention. The global-level at- tention can model the graphical structure of HKG using hypergraph dual-attention layers, while the local-level attention can learn the se- quential structure inside H-Facts via heteroge- neous self-attention layers. Experiment results indicate that HAHE achieves state-of-the-art performance in link prediction tasks on HKG standard datasets. In addition, HAHE addresses the issue of HKG multi-position prediction for the first time, increasing the applicability of the HKG link prediction task. Our code is publicly available1. 1 Introduction Knowledge graphs (KGs) are semantic networks that define entity relationships. Early KG re- search (Bordes et al., 2013; Sun et al., 2019; Bal- azevic et al., 2019) use binary relationships, of- ten expressed as a triple-based fact (subject, rela- tion, object). Yet, n-ary relational facts (containing more than two entities) are abundant in real-world KGs like Freebase (Bollacker et al., 2008) and Wikidata (Vrandeˇci´c and Krötzsch, 2014). Rosso et al. (2020) represent an n-ary relational fact as a ∗Corresponding author. 1https://github.com/LHRLAB/HAHE Barack Obama position held President of United States start time 20 January 2009 end time 20 January 2017 replaces George W. Bush series ordinal 44 replaced by Donald Trump Main Triple Qualifiers Barack Obama replaced George W. Bush as the 44th US president on 20 January 2009, and was replaced by Donald Trump on 20 January 2017. Figure 1: An example of hyper-relational fact structure. hyper-relational fact (H-Fact) consisting of a main triple(s,r,o) and several auxiliary attribute-value qualifiers {(ai :vi)}, and KGs composed of H- Facts are called hyper-relational knowledge graphs (HKGs). As shown in Figure 1, an H-fact can describe a real-world fact. Unlike traditional triple-based facts, H-Facts do not just raise the number of en- tities in facts from two to n. It structurally and ef- fectively represents the n-ary relational facts preva- lent in reality. Globally, it extends ordinary graph structure to hypergraph (Zhou et al., 2006) …

Content truncated to 500 words