Prompt-based Conservation Learning for Multi-hop Question Answering

Proceedings of the 29th International Conference on Computational Linguistics, pages 1791–1800 October 12–17, 2022. 1791 Prompt-based Conservation Learning for Multi-hop Question Answering Zhenyun Deng, Yonghua Zhu, Yang Chen, Qianqian Qi Michael Witbrock, Patricia Riddle School of Computer Science, University of Auckland, New Zealand {zden658, yzhu970, beryl413}@aucklanduni.ac.nz {yang.chen, m.witbrock, p.riddle}@auckland.ac.nz Abstract Multi-hop question answering (QA) requires reasoning over multiple documents to answer a complex question and provide interpretable supporting evidence. However, providing sup- porting evidence is not enough to demonstrate that a model has performed the desired reason- ing to reach the correct answer. Most existing multi-hop QA methods fail to answer a large fraction of sub-questions, even if their parent questions are answered correctly. In this pa- per, we propose the Prompt-based Conserva- tion Learning (PCL) framework for multi-hop QA, which acquires new knowledge from multi- hop QA tasks while conserving old knowledge learned on single-hop QA tasks, mitigating for- getting. Specifically, we first train a model on existing single-hop QA tasks, and then freeze this model and expand it by allocating addi- tional sub-networks for the multi-hop QA task. Moreover, to condition pre-trained language models to stimulate the kind of reasoning re- quired for specific multi-hop questions, we learn soft prompts for the novel sub-networks to perform type-specific reasoning. Experimen- tal results on the HotpotQA benchmark show that PCL is competitive for multi-hop QA and retains good performance on the corresponding single-hop sub-questions, demonstrating the ef- ficacy of PCL in mitigating knowledge loss by forgetting. 1 Introduction Multi-hop QA is a challenging task with the goals of reasoning over multiple scattered documents to predict an answer, and providing explanatory sup- porting evidence (Yang et al., 2018). By fine-tuning pre-trained language models (PLMs) with task- specific data, most existing multi-hop QA models have achieved good performance in both goals (Tu et al., 2020; Fang et al., 2020). Despite the success of fine-tuned PLMs on the multi-hop QA task, providing supporting evidence is not enough to demonstrate that a multi-hop QA Task1 Task2 Figure 1: An example of conservation learning based on a continual learning mechanism. The neurons on the left are devoted to Task1 (single-hop QA), and on the right (green) are a novel sub-network created for Task2 (multi-hop QA) that laterally connects to the trained Task1. By adding the sub- network, the model acquires new knowledge of Task2 while retaining knowledge learned in Task1, mitigating forgetting. model has performed the desired multi-hop reason- ing to reach the correct answer; it may instead have utilized reasoning shortcuts, having neglected to acquire and retain the single-hop reasoning knowl- edge essential to reliable interpretability (Jiang and Bansal, 2019). Previous work (Tang et al., 2021) has demonstrated that most existing multi-hop QA models with good performance fail to answer a large fraction of the sub-questions whose parent multi-hop questions can be answered correctly. Thus, it is necessary to understand the behaviour on each hop of the reasoning process and mitigate forgetting of the knowledge required for each hop in interpretable multi-hop QA. Doing …

Content truncated to 500 words