Knowledge Base Completion Meets Transfer Learning

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6521–6533 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6521 Knowledge Base Completion Meets Transfer Learning Vid Kocijan1, Thomas Lukasiewicz1,2 1University of Oxford 2Alan Turing Institute, London firstname.lastname@cs.ox.ac.uk Abstract The aim of knowledge base completion is to predict unseen facts from existing facts in knowledge bases. In this work, we intro- duce the first approach for transfer of knowl- edge from one collection of facts to another without the need for entity or relation match- ing. The method works for both canonicalized knowledge bases and uncanonicalized or open knowledge bases, i.e., knowledge bases where more than one copy of a real-world entity or relation may exist. Such knowledge bases are a natural output of automated information ex- traction tools that extract structured data from unstructured text. Our main contribution is a method that can make use of a large-scale pre-training on facts, collected from unstruc- tured text, to improve predictions on structured data from a specific domain. The introduced method is the most impactful on small datasets such as ReVerb20K, where we obtained 6% absolute increase of mean reciprocal rank and 65% relative decrease of mean rank over the previously best method, despite not relying on large pre-trained models like BERT. 1 Introduction A knowledge base (KB) is a collection of facts, stored and presented in a structured way that al- lows a simple use of the collected knowledge for applications. In this paper, a knowledge base is a finite set of triples ⟨h, r, t⟩, where h and t are head and tail entities, while r is a binary relation between them. Manually constructing a knowledge base is tedious and requires a large amount of labor. To speed up the process of construction, facts can be extracted from unstructured text automatically, using, e.g., open information extraction (OIE) tools, such as ReVerb (Fader et al., 2011) or more recent neural approaches (Stanovsky et al., 2018; Hohe- necker et al., 2020). Alternatively, missing facts can be inferred from existing ones using knowl- edge base completion (KBC) algorithms, such as the United Kingdom UK Alan Turing Europe lived in part of lived in (?) Figure 1: An example of a small knowledge base with the fact whether Alan Turing lived in Europe missing. If the knowledge base is canonicalized, “the United Kingdom” and “UK” are known to be the same entity. If the knowledge base is uncanonicalized or open, this information may not be given. ConvE (Dettmers et al., 2018), TuckER (Balaževi´c et al., 2019), or 5⋆E (Nayyeri et al., 2021). It is desirable to use both OIE and knowledge base completion approaches to automatically con- struct KBs. However, automatic extractions from text yield uncanonicalized entities and relations. An entity such as “the United Kingdom” may also appear as “UK”, and a relation such as “located at” may also appear as “can be found in”. If we fail to connect these occurrences and treat them as distinct entities and …

Content truncated to 500 words