Edge: Enriching Knowledge Graph Embeddings with External Text

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2767–2776 June 6–11, 2021. ©2021 Association for Computational Linguistics 2767 EDGE: Enriching Knowledge Graph Embeddings with External Text Saed Rezayi1, Handong Zhao2, Sungchul Kim2, Ryan A. Rossi2, Nedim Lipka2, and Sheng Li1 1Department of Computer Science, University of Georgia, Athens, GA, USA 2Adobe Research, San Jose, CA, USA {saedr,sheng.li}@uga.edu {hazhao,sukim,ryrossi,lipka}@adobe.com Abstract Knowledge graphs suffer from sparsity which degrades the quality of representations gener- ated by various methods. While there is an abundance of textual information throughout the web and many existing knowledge bases, aligning information across these diverse data sources remains a challenge in the litera- ture. Previous work has partially addressed this issue by enriching knowledge graph enti- ties based on “hard” co-occurrence of words present in the entities of the knowledge graphs and external text, while we achieve “soft” aug- mentation by proposing a knowledge graph enrichment and embedding framework named EDGE. Given an original knowledge graph, we first generate a rich but noisy augmented graph using external texts in semantic and structural level. To distill the relevant knowledge and suppress the introduced noise, we design a graph alignment term in a shared embedding space between the original and augmented graph. To enhance the embedding learning on the augmented graph, we further regularize the locality relationship of target entity based on negative sampling. Experimental results on four benchmark datasets demonstrate the ro- bustness and effectiveness of EDGE in link pre- diction and node classification. 1 Introduction Knowledge Graph (KG)1 embedding learning has been an emerging research topic in natural lan- guage processing, which aims to learn a low di- mensional latent vector for every node. One major challenge is sparsity. Knowledge graphs are of- ten incomplete, and it is a challenge to generate low-dimensional representations from a graph with many missing edges. To mitigate this issue, auxil- 1Knowledge graph usually represents a heterogeneous multigraph whose nodes and relations can have different types. However in the work, we follow (Kartsaklis et al., 2018), con- sider knowledge graph enrichment problem where only one relation type (connected or not) appears. 1 protective body wound arthropod “Insect bite, nonvenomous, of foot and toe infected…” “Nonvenomous insect bite of hip without infection …” augmentation distillation Figure 1: An example illustrating the original (left) and augmented knowledge graphs (right). Red nodes are knowledge graph entities and small blue nodes are tex- tual nodes obtained from the external text. In augmen- tation process, a new set of keywords are discovered and attached to the original entities. To keep the aug- mented graph semantically close to the original graph, a backward pass of knowledge distillation is achieved by the proposed graph alignment. iary texts that are easily accessible have been popu- larly exploited for enhancing the KG (as illustrated in Figure 1). More specifically, given that KG en- tities contain textual features, we can link them to an auxiliary source of knowledge, e.g., WordNet, and therefore …

Content truncated to 500 words