Interpretable Proof Generation via Iterative Backward Reasoning

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2968 - 2981 July 10-15, 2022 ©2022 Association for Computational Linguistics Interpretable Proof Generation via Iterative Backward Reasoning Hanhao Qu1 Yu Cao3 Jun Gao1 Liang Ding3,4 Ruifeng Xu1,2∗ 1Harbin Institute of Technology, Shenzhen 2Peng Cheng Laboratory {hhqu0917,jgao95}@stu.hit.edu.cn xuruifeng@hit.edu.cn 3The University of Sydney, Australia 4JD Explore Academy {ycao8647,ldin3097}@uni.sydney.edu.au Abstract We present IBR, an Iterative Backward Reason- ing model to solve the proof generation tasks on rule-based Question Answering (QA), where models are required to reason over a series of textual rules and facts to find out the related proof path and derive the final answer. We handle the limitations of existed works in two folds: 1) enhance the interpretability of rea- soning procedures with detailed tracking, by predicting nodes and edges in the proof path iteratively backward from the question; 2) pro- mote the efficiency and accuracy via reason- ing on the elaborate representations of nodes and history paths, without any intermediate texts that may introduce external noise dur- ing proof generation. There are three main modules in IBR, QA and proof strategy pre- diction to obtain the answer and offer guid- ance for the following procedure; parent node prediction to determine a node in the existing proof that a new child node will link to; child node prediction to find out which new node will be added to the proof. Experiments on both synthetic and paraphrased datasets demon- strate that IBR has better in-domain perfor- mance as well as cross-domain transferability than several strong baselines. Our code and models are available at https://github. com/find-knowledge/IBR. 1 Introduction Endowing machines with reasoning capabilities is a longstanding problem (Newell and Simon, 1956) in the field of AI. Though existing tasks such as multi-hop QA (Yang et al., 2018; Welbl et al., 2018) or logical-reasoning QA (Yu et al., 2020; Dua et al., 2019) impose a higher requirement on the reason- ing capabilities, they usually just request for an answer without the reasoning procedure that would make it interpretable. Recently, Clark et al. (2020) proposed new datasets and tasks for interpretable ∗Corresponding author Facts: F1: Anne is blue. F2: Anne is rough. F9: Fiona is rough. F10: Harry is blue. Rules: R1: All rough, blue people are cold. R2: All cold people are round. R6: If Harry is blue then Harry is rough. R7: Quiet people are round. R8: If someone is round and not cold then they are quiet. Q1: Harry is round. A1: True Proof: F10 R2 R1 Q1 R6 F10 Q2: Fiona is not cold. A2: True Proof: Q2 R1 FAIL R7 … wrong branch F9 redundant branch … … Figure 1: Illustration of generating proof iteratively. Re- garding the proof path as a graph, and using the question as the initial node, other nodes and edges will be added step by step. (The gold proof is the obtained path in a re- verse order exclude the question). The main challenges …

Content truncated to 500 words