Case-based Reasoning for Natural Language Queries over Knowledge Bases

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9594–9611 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9594 Case-Based Reasoning for Natural Language Queries over Knowledge Bases Rajarshi Das1, Manzil Zaheer3, Dung Thai1, Ameya Godbole1, Ethan Perez2, Jay-Yoon Lee1, Lizhen Tan4, Lazaros Polymenakos5∗, Andrew McCallum1 1University of Massachusetts, Amherst, 2New York University 3Google Research, 4Amazon, 5EY, Greece {rajarshi, dthai, agodbole, jaylee, mccallum}@cs.umass.edu Abstract It is often challenging to solve a complex prob- lem from scratch, but much easier if we can access other similar problems with their solu- tions — a paradigm known as case-based rea- soning (CBR). We propose a neuro-symbolic CBR approach (CBR-KBQA) for question an- swering over large knowledge bases. CBR- KBQA consists of a nonparametric memory that stores cases (question and logical forms) and a parametric model that can generate a logical form for a new question by retrieving cases that are relevant to it. On several KBQA datasets that contain complex questions, CBR- KBQA achieves competitive performance. For example, on the COMPLEXWEBQUESTIONS dataset, CBR-KBQA outperforms the current state of the art by 11% on accuracy. Further- more, we show that CBR-KBQA is capable of using new cases without any further training: by incorporating a few human-labeled exam- ples in the case memory, CBR-KBQA is able to successfully generate logical forms containing unseen KB entities as well as relations. 1 Introduction Humans often solve a new problem by recollecting and adapting the solution to multiple related prob- lems that they have encountered in the past (Ross, 1984; Lancaster and Kolodner, 1987; Schmidt et al., 1990). In classical artificial intelligence (AI), case-based reasoning (CBR) pioneered by Schank (1982), tries to incorporate such model of reason- ing in AI systems (Kolodner, 1983; Rissland, 1983; Leake, 1996). A sketch of a CBR system (Aamodt and Plaza, 1994) comprises of — (i) a retrieval module, in which ‘cases’ that are similar to the given problem are retrieved, (ii) a reuse module, ∗Now director of AI at EY CESA, Greece. Work done while at Amazon where the solutions of the retrieved cases are re- used to synthesize a new solution. Often, the new solution does not work and needs more revision, which is handled by a (iii) revise module. In its early days, the components of CBR were implemented with symbolic systems, which had their limitations. For example, finding similar cases and synthesizing new solutions from them is a chal- lenging task for a CBR system implemented with symbolic components. However, with the recent advancements in representation learning (LeCun et al., 2015), the performance of ML systems have improved substantially on a range of practical tasks. Given a query, CBR-KBQA uses a neural re- triever to retrieve other similar queries (and their logical forms) from a case memory (e.g. training set). Next, CBR-KBQA generates a logical form for the given query by learning to reuse various com- ponents of the logical forms of the retrieved cases. However, often the generated logical form does not produce the …

Content truncated to 500 words