Modeling Multi-hop Question Answering as Single Sequence Prediction

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 974 - 990 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Modeling Multi-hop Question Answering as Single Sequence Prediction Semih Yavuz Kazuma Hashimoto Yingbo Zhou Nitish Shirish Keskar Caiming Xiong Salesforce Research {syavuz,k.hashimoto,yingbo.zhou,nkeskar,cxiong}@salesforce.com Abstract Fusion-in-decoder (FID) (Izacard and Grave, 2021) is a generative question answering (QA) model that leverages passage retrieval with a pre-trained transformer and pushed the state of the art on single-hop QA. However, the com- plexity of multi-hop QA hinders the effective- ness of the generative QA approach. In this work, we propose a simple generative approach (PATHFID) that extends the task beyond just an- swer generation by explicitly modeling the rea- soning process to resolve the answer for multi- hop questions. By linearizing the hierarchical reasoning path of supporting passages, their key sentences, and finally the factoid answer, we cast the problem as a single sequence predic- tion task. To facilitate complex reasoning with multiple clues, we further extend the unified flat representation of multiple input documents by encoding cross-passage interactions. Our ex- tensive experiments demonstrate that PATHFID leads to strong performance gains on two multi- hop QA datasets: HotpotQA and IIRC. Besides the performance gains, PATHFID is more inter- pretable, which in turn yields answers that are more faithfully grounded to the supporting pas- sages and facts compared to the baseline FID model. 1 Introduction Leveraging knowledge to make complex reasoning has been a fundamental problem of artificial intel- ligence. Open-domain question answering (QA) (Voorhees, 1999) is an integral part of such a line of research with impactful applications (Esteva et al., 2020; Zhang et al., 2020), where the task is to answer general domain questions by gathering evi- dence from a large collection of documents. While super-human level performance has been achieved on single-passage reading comprehension dataset like SQuAD (Rajpurkar et al., 2016), open-domain QA still has a long way to go, especially for ques- tions requiring more complex reasoning. The main challenge in the task of complex QA, namely multi- hop QA, is that it requires a QA system to combine multiple pieces of evidence from multiple docu- ments (Welbl et al., 2018; Talmor and Berant, 2018; Yang et al., 2018). Even for single-hop QA, it has been shown challenging for extractive QA models to effectively aggregate evidence from the com- bined pool of multiple passages, which has been the focus of recent work (Clark and Gardner, 2018; Min et al., 2019; Guu et al., 2020). Recent work (Lewis et al., 2020b; Min et al., 2020) has demonstrated the promise of a genera- tive approach at combining evidences from mul- tiple passages for answer generation. Thanks to large pre-trained transformers like T5 (Raffel et al., 2020), Izacard and Grave (2021) introduced fusion-in-decoder (FID) that leverages passage re- trieval with generative models for open-domain QA, achieving state-of-the-art scores across several single-hop QA benchmarks. However, we observe that the success of the FID model does not extend to …

Content truncated to 500 words