M3: A Multi-View Fusion and Multi-Decoding Network for Multi-Document Reading Comprehension

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1450 - 1461 December 7-11, 2022 ©2022 Association for Computational Linguistics M3: A Multi-View Fusion and Multi-Decoding Network for Multi-Document Reading Comprehension Liang Wen†,‡ Houfeng Wang† Yingwei Luo†,‡ Xiaolin Wang†,‡ †School of Computer Science, Peking University, China ‡Peng Cheng Laboratory, Shenzhen, China {yuco,wanghf,lyw,wxl}@pku.edu.cn Abstract Multi-document reading comprehension task requires collecting evidences from different documents for answering questions. Previous research works either use the extractive mod- eling method to naively integrate the scores from different documents on the encoder side or use the generative modeling method to col- lect the clues from different documents on the decoder side individually. However, any sin- gle modeling method cannot make full of the advantages of both. In this work, we pro- pose a novel method that tries to employ a multi-view fusion and multi-decoding mecha- nism to achieve it. For one thing, our approach leverages question-centered fusion mechanism and cross-attention mechanism to gather fine- grained fusion of evidence clues from differ- ent documents in the encoder and decoder concurrently. For another, our method si- multaneously employs both the extractive de- coding approach and the generative decod- ing method to effectively guide the training process. Compared with existing methods, our method can perform both extractive de- coding and generative decoding independently and optionally. Our experiments on two mainstream multi-document reading compre- hension datasets (Natural Questions and Triv- iaQA) demonstrate that our method can pro- vide consistent improvements over previous state-of-the-art methods. 1 Introduction Open domain question answering (QA) aims to produce an answer for a given question using a large text corpus source such as Wikipedia. One of the typical approaches to open domain QA follows the retriever-reader framework (Chen et al., 2017; Karpukhin et al., 2020; Izacard and Grave, 2021b), where a retriever first identifies the most relevant documents, then a reader understands the retrieved documents and produces an answer. In this work, we focus on improving the effectiveness of the reader, whose goal is to efficiently aggregate and combine evidence from multiple documents for better answering questions, which is also known as multi-document reading comprehension (Hu et al., 2019). Depending on the difference of decoding ap- proaches, recent works on multi-document read- ing comprehension could be mainly divided into two categories: extractive approaches (Lee et al., 2019; Karpukhin et al., 2020; Guu et al., 2020) and generative approaches (Lewis et al., 2020b; Izacard and Grave, 2021b; Yu et al., 2022). To pro- duce an answer, extractive approaches make pre- dictions by extracting a contiguous span from the given evidence documents as answer. Since it is not straightforward for extractive models to aggregate and combine evidence from multiple documents (or paragraphs), various techniques have been put forward to address this limitation (Clark and Gard- ner, 2018; Wang et al., 2018; Min et al., 2019a). Besides, another drawback of extractive models is that they can’t produce an answer string if it does not contained in the given evidence documents. Izacard and Grave …

Content truncated to 500 words