Probabilistic Graph Reasoning for Natural Proof Generation

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3140–3151 August 1–6, 2021. ©2021 Association for Computational Linguistics 3140 Probabilistic Graph Reasoning for Natural Proof Generation Changzhi Sun†*, Xinbo Zhang†*, Jiangjie Chen†§, Chun Gan†¶, Yuanbin Wu‡, Jiaze Chen†, Hao Zhou†, and Lei Li† †ByteDance AI Lab §School of Computer Science, Fudan University ¶Math Department, University of Wisconsin–Madison ‡School of Computer Science and Technology, East China Normal University {sunchangzhi,zhangxinbo.freya}@bytedance.com {chenjiaze,zhouhao.nlp,lileilab}@bytedance.com jjchen19@fudan.edu.cn, cgan5@wisc.edu, ybwu@cs.ecnu.edu.cn Abstract In this paper, we investigate the problem of reasoning over natural language statements. Prior neural based approaches do not explic- itly consider the inter-dependency among an- swers and their proofs. In this paper, we pro- pose PROBR, a novel approach for joint an- swer prediction and proof generation. PROBR defines a joint probabilistic distribution over all possible proof graphs and answers via an in- duced graphical model. We then optimize the model using variational approximation on top of neural textual representation. Experiments on multiple datasets under diverse settings (fully supervised, few-shot and zero-shot eval- uation) verify the effectiveness of PROBR, e.g., achieving 10%-30% improvement on QA ac- curacy in few/zero-shot evaluation. Our codes and models can be found at https://github.com/ changzhisun/PRobr/. 1 Introduction Automatic reasoning over explicitly provided knowledge has been a persistent goal of AI (Newell and Simon, 1956; McCarthy et al., 1960). Early ap- proaches focus on reasoning over formal (logical or probabilistic) representations. However, automati- cally constructing and reasoning over formal rep- resentations remain challenging. To bypass these challenges, in this work, we investigate reasoning over natural language statements instead of formal representations. Given a set of facts and rules and a query (ex- pressed in natural language), we aim to predict the answer and provide proof to prove or disprove the query. For example, in Figure 1, there are two facts, six rules and two queries, each of which is expressed by natural language. To predict the true/false of each query, starting from the facts, we need to reason deductively by applying given rules ∗Equal contribution. NAF NAF F1 R1 R6 R3 R4 F2 Facts : F1: The circuit includes the battery. F2: The wire is metal. Rules : R1: If the circuit includes the battery and the battery is not flat then the circuit is powered. R2: If the circuit includes the switch and the switch is on then the circuit is complete. R3: If the circuit does not have the switch then the circuit is complete. R4: If the wire is metal then the wire is conducting. R5: If the wire is plastic then the wire is not conducting. R6: If the circuit is powered and the circuit is complete and the wire is conducting then the current runs through the circuit. Q1 : The wire is conducting. A1: True Proof ： Q2 : The current does not run through the circuit. A2: False Proof ： F2 R4 Figure 1: An example of reasoning over natural lan- guage statements. The goal is to predict the answer (true/false) and generate the …

Content truncated to 500 words