Natural Language Deduction with Incomplete Information

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8230 - 8258 December 7-11, 2022 ©2022 Association for Computational Linguistics Natural Language Deduction with Incomplete Information Zayne Sprague Kaj Bostrom Swarat Chaudhuri Greg Durrett Department of Computer Science The University of Texas at Austin zaynesprague@utexas.edu, {kaj,swarat,gdurrett}@cs.utexas.edu Abstract A growing body of work studies how to answer a question or verify a claim by generating a natural language “proof”: a chain of deductive inferences yielding the answer based on a set of premises. However, these methods can only make sound deductions when they follow from evidence that is given. We propose a new system that can handle the underspecified setting where not all premises are stated at the outset; that is, additional assumptions need to be materialized to prove a claim. By using a natural language generation model to abductively infer a premise given another premise and a conclusion, we can impute missing pieces of evidence needed for the conclusion to be true. Our system searches over two fringes in a bidirectional fashion, interleaving deductive (forward-chaining) and abductive (backward- chaining) generation steps. We sample multiple possible outputs for each step to achieve coverage of the search space, at the same time ensuring correctness by filtering low- quality generations with a round-trip validation procedure. Results on a modified version of the EntailmentBank dataset and a new dataset called Everyday Norms: Why Not? show that abductive generation with validation can recover premises across in- and out-of-domain settings.1 1 Introduction Substantial prior work in domains like question answering (Rajpurkar et al., 2016; Yang et al., 2018; Kwiatkowski et al., 2019), textual entailment (Bowman et al., 2015; Williams et al., 2018; Nie et al., 2020), and other types of reasoning (Clark et al., 2021; Dalvi et al., 2021) deals with making inferences from stated information, where we draw conclusions and answer questions 1Code and data publicly available at //github.com/Zayne-sprague/Natural_Language_https: Deduction_with_Incomplete_Information.git An animal requires oxygen to breathe. A human is a kind of animal Astronauts are human. An astronaut is a kind of animal. An astronaut requires oxygen to breathe. Animals require oxygen to breathe. Deduc&on Abduc&on Evidence Goal Entailment:4did4we4recover4the4missing4premise? Held-out)premise Figure 1: An example of deductive (previous work) and abductive (our work) reasoning used together to search for missing evidence needed to entail a goal in a depth 2 tree from EntailmentBank. based on textual context provided directly to a model. However, a growing body of research studies the problem of reasoning given incomplete information, especially for tasks labeled as commonsense reasoning (Talmor et al., 2019; Rajani et al., 2019). Current approaches in these domains often work through latent reasoning by large language models (Lourie et al., 2021), with only a few explicitly materializing the missing knowledge (Bosselut et al., 2019; Bhagavatula et al., 2020; Arabshahi et al., 2021; Liu et al., 2022; Katz et al., 2022). However, making knowledge explicit is critical to make reasoning processes explainable: it allows users to critique those explanations and allows systems to reuse inferred …

Content truncated to 500 words