Transformer-based Entity Typing in Knowledge Graphs

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5988 - 6001 December 7-11, 2022 ©2022 Association for Computational Linguistics Transformer-based Entity Typing in Knowledge Graphs Zhiwei Hu♣ Víctor Gutiérrez-Basulto♢ Zhiliang Xiang♢ Ru Li♣∗ Jeff Z. Pan♠∗ ♣ School of Computer and Information Technology, Shanxi University, China ♦ School of Computer Science and Informatics, Cardiff University, UK ♠ ILCC, School of Informatics, University of Edinburgh, UK ♣ zhiweihu@whu.edu.cn,liru@sxu.edu.cn ♦{gutierrezbasultov,xiangz6}@cardiff.ac.uk ♠j.z.pan@ed.ac.uk Abstract We investigate the knowledge graph entity typ- ing task which aims at inferring plausible en- tity types. In this paper, we propose a novel Transformer-based Entity Typing (TET) ap- proach, effectively encoding the content of neighbors of an entity. More precisely, TET is composed of three different mechanisms: a local transformer allowing to infer missing types of an entity by independently encoding the information provided by each of its neigh- bors; a global transformer aggregating the in- formation of all neighbors of an entity into a single long sequence to reason about more com- plex entity types; and a context transformer integrating neighbors content based on their contribution to the type inference through in- formation exchange between neighbor pairs. Furthermore, TET uses information about class membership of types to semantically strengthen the representation of an entity. Experiments on two real-world datasets demonstrate the su- perior performance of TET compared to the state-of-the-art. 1 Introduction A knowledge graph (KG) (Pan et al., 2016) is a multi-relational graph encoding factual knowledge, with the form (h, r, t) where h, t are the head and tail entities connected via the relation r. In this paper, we consider KGs with minimal schema information, i.e., those containing entity type as- sertions, as the only schema information, of the form (e, has_type, c) stating that the entity e has type c; e.g., to capture that Barack Obama has type President. Entity type knowledge is widely used in NLP tasks, e.g., in relation extraction (Liu et al., 2014), entity and relation linking (Gupta et al., 2017; Pan et al., 2019), question answering (ElSahar et al., 2018; Hu et al., 2022), and fine- grained entity typing on text (Onoe et al., 2021; Qian et al., 2021; Liu et al., 2021). However, entity ∗Contact Authors Barack Obama A Promised Land 1961 U.S Democratic Party Juris Doctor Columbia University Book 20th-century American writers Politician President Columbia University Alumni American Legal Scholars Bachelor of Science Relational neighbors Type neighbors degree award write Entity Type Missing type KG relation Type link Missing type link Figure 1: A KG with its entity type information. types are far from complete, since in real-world applications they are continuously emerging. For example, about 10% of entities in FB15k (Bordes et al., 2013) have the type /music/artist, but do not have /people/person (Moon et al., 2017). In light of this, it has been recently investigated the Knowledge Graph Entity Typing (KGET) task, aiming at inferring missing entity types in a KG. Most existing approaches to KGET use methods based on either embeddings …

Content truncated to 500 words