RankQA: Neural Question Answering with Answer Re-Ranking

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6076–6085 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 6076 RankQA: Neural Question Answering with Answer Re-Ranking Bernhard Kratzwald♠ Anna Eigenmann♦ Stefan Feuerriegel♠ ♠ Chair of Management Information Systems, ETH Zurich ♦ Department of Mathematics, ETH Zurich {bkratzwald, eianna, sfeuerriegel}@ethz.ch Abstract The conventional paradigm in neural question answering (QA) for narrative content is limited to a two-stage process: first, relevant text pas- sages are retrieved and, subsequently, a neural network for machine comprehension extracts the likeliest answer. However, both stages are largely isolated in the status quo and, hence, information from the two phases is never prop- erly fused. In contrast, this work proposes RankQA1: RankQA extends the conventional two-stage process in neural QA with a third stage that performs an additional answer re- ranking. The re-ranking leverages different features that are directly extracted from the QA pipeline, i. e., a combination of retrieval and comprehension features. While our inten- tionally simple design allows for an efficient, data-sparse estimation, it nevertheless outper- forms more complex QA systems by a signif- icant margin: in fact, RankQA achieves state- of-the-art performance on 3 out of 4 bench- mark datasets. Furthermore, its performance is especially superior in settings where the size of the corpus is dynamic. Here the answer re- ranking provides an effective remedy against the underlying noise-information trade-off due to a variable corpus size. As a consequence, RankQA represents a novel, powerful, and thus challenging baseline for future research in content-based QA. 1 Introduction Question answering (QA) has recently experi- enced considerable success in variety of bench- marks due to the development of neural QA (Chen et al., 2017; Wang et al., 2018). These systems largely follow a two-stage process. First, a mod- ule for information retrieval selects text passages which appear relevant to the query from the cor- 1Code is available from https://github.com/ bernhard2202/rankqa pus. Second, a module for machine comprehen- sion extracts the final answer, which is then re- turned to the user. This two-stage process is neces- sary for condensing the original corpus to passages and eventually answers; however, the dependence limits the extent to which information is passed on from one stage to the other. Extensive efforts have been made to facilitate better information flow between the two stages. These works primarily address the interface be- tween the stages (Lee et al., 2018; Lin et al., 2018), i. e., which passages and how many of them are forwarded from information retrieval to ma- chine comprehension. For instance, the QA per- formance is dependent on the corpus size and the number of top-n passages that are fed into the module for machine comprehension (Kratzwald and Feuerriegel, 2018). Nevertheless, machine comprehension in this approach makes use of only limited information (e. g., it ignores the confi- dence or similarity information computed during retrieval). State-of-the-art approaches for selecting better answers engineer additional features within the machine comprehension model with the implicit goal …

Content truncated to 500 words