Exploring the Limits of Few-Shot Link Prediction in Knowledge Graphs

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2816–2822 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2816 Exploring the Limits of Few-Shot Link Prediction in Knowledge Graphs Dora Jambor1,2, Komal Teru1,2, Joelle Pineau1,2,3, William L. Hamilton1,2 1 School of Computer Science, McGill University, Canada 2 Quebec AI Institute (Mila), Canada 3 Facebook AI Research (FAIR) {dora.jambor, komal.teru}@mail.mcgill.ca {jpineau, wlh}@cs.mcgill.ca Abstract Real-world knowledge graphs are often char- acterized by low-frequency relations—a chal- lenge that has prompted an increasing interest in few-shot link prediction methods. These methods perform link prediction for a set of new relations, unseen during training, given only a few example facts of each relation at test time. In this work, we perform a sys- tematic study on a spectrum of models de- rived by generalizing the current state of the art for few-shot link prediction, with the goal of probing the limits of learning in this few- shot setting. We find that a simple zero-shot baseline—which ignores any relation-specific information—achieves surprisingly strong per- formance. Moreover, experiments on carefully crafted synthetic datasets show that having only a few examples of a relation fundamen- tally limits models from using fine-grained structural information and only allows for ex- ploiting the coarse-grained positional informa- tion of entities. Together, our findings chal- lenge the implicit assumptions and inductive biases of prior work and highlight new direc- tions for research in this area. 1 Introduction A knowledge graph (KG) is a multi-relational graph that offers a structured way to organize facts about the world. Encoder-decoder approaches are com- monly used to predict new facts from existing ones where entities and relations are embedded in a low- dimensional vector space via an encoder, to then score the likelihood of observing a new fact via a decoder (Nickel et al., 2015; Bordes et al., 2013; Trouillon et al., 2017; Dettmers et al., 2018). It is well known that the performance of these methods can significantly drop when predicting for relations that are only observed in a few ex- ample facts. However, link prediction for these low-frequency relations is very important, as not only are these relations abundant in most knowl- edge graphs, they are also key for knowledge graph ���������(Richard�Feynman,���born_in,���United�States) �������(Albert�Einstein,���born_in,���Germany) (Galileo�Galilei,���born_in,���Italy) Knowledge Transfer Support set: K example facts Query set: Test fact ���������(Paul�Erdős,���born_in,���?) Figure 1: The few-shot link prediction task completion tasks where new relations may appear after model training. To study this low-frequency regime, Xiong et al. (2018) created the Nell-One and Wiki-One bench- marks where the task is to predict new facts for a set of new relations at test time, where each relation is only observed a few times (as specified by some small fixed number K). Previous approaches have shown promising results using metric-based (Xiong et al., 2018) and gradient based meta-learning tech- niques (Chen et al., 2019). However, we argue that these models are limited by the current task for- mulation to only exploit coarse-grained positional signals (i.e., nodes …

Content truncated to 500 words