Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2285–2295 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2285 Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension Minghao Hu, Yuxing Peng, Zhen Huang, Dongsheng Li National University of Defense Technology, Changsha, China {huminghao09,pengyuxing,huangzhen,dsli}@nudt.edu.cn Abstract This paper considers the reading comprehen- sion task in which multiple documents are given as input. Prior work has shown that a pipeline of retriever, reader, and reranker can improve the overall performance. How- ever, the pipeline system is inefficient since the input is re-encoded within each module, and is unable to leverage upstream compo- nents to help downstream training. In this work, we present RE3QA, a unified question answering model that combines context re- trieving, reading comprehension, and answer reranking to predict the final answer. Unlike previous pipelined approaches, RE3QA shares contextualized text representation across dif- ferent components, and is carefully designed to use high-quality upstream outputs (e.g., re- trieved context or candidate answers) for di- rectly supervising downstream modules (e.g., the reader or the reranker). As a result, the whole network can be trained end-to-end to avoid the context inconsistency problem. Ex- periments show that our model outperforms the pipelined baseline and achieves state-of- the-art results on two versions of TriviaQA and two variants of SQuAD. 1 Introduction Teaching machines to read and comprehend text is a long-term goal of natural language process- ing. Despite recent success in leveraging reading comprehension (RC) models to answer questions given a related paragraph (Wang et al., 2017; Hu et al., 2018; Yu et al., 2018), extracting answers from documents or even a large corpus of text (e.g., Wikipedia or the whole web) remains to be an open challenge. This paper considers the multi- document RC task (Joshi et al., 2017), where the system needs to, given a question, identify the an- swer from multiple evidence documents. Unlike single-pargraph settings (Rajpurkar et al., 2016), this task typically involves a retriever for select- ing few relevant document content (Chen et al., 2017), a reader for extracting answers from the retrieved context (Clark and Gardner, 2018), and even a reranker for rescoring multiple candidate answers (Bogdanova and Foster, 2016). Previous approaches such as DS-QA (Lin et al., 2018) and R3 (Wang et al., 2018a) consist of sep- arate retriever and reader models that are jointly trained. Wang et al. (2018d) further propose to rerank multiple candidates for verifying the fi- nal answer. Wang et al. (2018b) investigate the full retrieve-read-rerank process by constructing a pipeline system that combines an information re- trieval (IR) engine, a neural reader, and two kinds of answer rerankers. Nevertheless, the pipeline system requires re-encoding inputs for each sub- task, which is inefficient for large RC tasks. More- over, as each model is trained independently, high- quality upstream outputs can not benefit down- stream modules. For example, as the training pro- ceeds, a neural retriever is able to provide more relevant context than an IR engine (Htut …

Content truncated to 500 words