Logical Neural Networks for Knowledge Base Completion with Embeddings & Rules

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3863 - 3875 December 7-11, 2022 ©2022 Association for Computational Linguistics Logical Neural Networks for Knowledge Base Completion with Embeddings & Rules Prithviraj Sen∗ Amazon Breno W. S. R. de Carvalho IBM Research Ibrahim Abdelaziz IBM Research Pavan Kapanipathi IBM Research Salim Roukos IBM Research Alexander Gray IBM Research Abstract Knowledge base completion (KBC) has bene- fitted greatly by learning explainable rules in an human-interpretable dialect such as first-order logic. Rule-based KBC has so far, mainly fo- cussed on learning one of two types of rules: conjunction-of-disjunctions and disjunction-of- conjunctions. We qualitatively show, via exam- ples, that one of these has an advantage over the other when it comes to achieving high qual- ity KBC. To the best of our knowledge, we are the first to propose learning both kinds of rules within a common framework. To this end, we propose to utilize logical neural networks (LNN) (Riegel et al., 2020), a powerful neuro- symbolic AI framework that can express both kinds of rules and learn these end-to-end using gradient-based optimization. Our in-depth ex- periments show that our LNN-based approach to learning rules for KBC leads to roughly 10% relative improvements, if not more, over SotA rule-based KBC methods. Moreover, by show- ing how to combine our proposed methods with knowledge graph embeddings we further achieve additional 7.5% relative improvement. 1 Introduction Knowledge bases (KB), e.g. Freebase (Bollacker et al., 2008), are inherently incomplete thus tech- niques have been proposed to identify facts missing from a KB. Knowledge base completion (KBC) can be broadly classified into knowledge graph em- beddings (KGE) and rule-based knowledge base completion (rule-based KBC). While KGE has achieved highly accurate KBC by learning a low dimensional hyperspace, rule-based KBC offers su- perior explainability by learning rules in a human- interpretable dialect such as first-order logic (FOL). In this work, we take a closer look at rule- based KBC. A knowledge base consists of facts denoted by triples of the form ⟨h, r, t⟩ where ∗Work done while first author was at IBM Research. ∀P, N∃L: citizenOf(P, N)←bornIn(P, C)∧ partOf(C, N) Al Pacino New York City U. S. bornIn partOf Figure 1: An FOL rule that infers a missing fact. tail t and head h denote entities while r de- notes a relation. Figure 1 shows a KB contain- ing two facts ⟨Al Pacino, bornIn, New York City⟩, ⟨New York City, partOf, U. S.⟩ denoted by (solid) directed edges. The goal of rule-based KBC is to infer missing facts using FOL rules such as the one shown on top of Figure 1 that combines a per- son P’s birth city C and the country in which C is present in to ascertain P’s citizenship. By substitut- ing P/Al Pacino, C/New York City, and N/U. S. into the rule we can infer that Al Pacino is a U. S. citizen which is a fact missing from the KB (de- noted by a dashed edge in …

Content truncated to 500 words