Out-of-Sample Representation Learning for Knowledge Graphs

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2657–2666 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2657 Out-of-Sample Representation Learning for Knowledge Graphs Marjan Albooyeh, Rishab Goel, and Seyed Mehran Kazemi Borealis AI, Montreal, Canada {marjan.albooyeh, rishab.goal, mehran.kazemi}@borealisai.com Abstract Many important problems can be formulated as reasoning in knowledge graphs. Represen- tation learning has proved extremely effective for transductive reasoning, in which one needs to make new predictions for already observed entities. This is true for both attributed graphs (where each entity has an initial feature vec- tor) and non-attributed graphs (where the only initial information derives from known rela- tions with other entities). For out-of-sample reasoning, where one needs to make predic- tions for entities that were unseen at train- ing time, much prior work considers attributed graph. However, this problem is surprisingly under-explored for non-attributed graphs. In this paper, we study the out-of-sample repre- sentation learning problem for non-attributed knowledge graphs, create benchmark datasets for this task, develop several models and base- lines, and provide empirical analyses and com- parisons of the proposed models and baselines. 1 Introduction Multi-relational graphs are a prevalent form of graphs where each edge has a label and a direc- tion associated with it. Many prediction problems can be formulated as reasoning within a multi- relational graph. For example, Figure 1 depicts a job recommendation system that has been formu- lated in these terms. A notable example of multi- relational graphs is knowledge graphs (KGs) with several applications in natural language processing and information retrieval including search, ques- tion answering and commonsense reasoning. Much prior work has considered transductive KG reason- ing in which predictions are made at test time for only those entities that were observed during train- ing. These are known as in-sample entities. In Figure 1, predicting if A1 is expert in S2 is an ex- ample of transductive reasoning. C1 J1 S1 A1 S2 Current_Comp Expert_In Posted_By Requires Child_Of C2 A2 J2 Fit_For T2 Job_Title S3 A3 T1 Jnew Posted_By Job_Title Current_Title Requires Requires Expert_In ? Expert_In Fit_For ? Figure 1: An example of a multi-relational graph for a job recommendation system is presented on the left side of the dashed blue line where the vertices Ai, Ci, Si, Ji and Ti represent applicants, companies, skills, job postings, and titles respectively. Predicting whether A1 is expert in S2 is an example of transductive reason- ing. Jnew represents an out-of-sample entity that has not been observed during training. Predicting whether A3 is a good fit for Jnew based on the relations of Jnew observed during test time (red arrows) is an example of out-of-sample reasoning. Conversely, we consider out-of-sample KG rea- soning. We make predictions for previously unseen or out-of-sample entities based on their relations with the in-sample entities. This is more challeng- ing than transductive reasoning as it requires gen- eralizing to unseen entities. In Figure 1, predicting whether A3 is a good fit for the previously unseen job posting Jnew given Jnew’s relations …

Content truncated to 500 words