Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4514–4525 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4514 Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering Vikas Yadav, Steven Bethard, Mihai Surdeanu University of Arizona, Tucson, AZ, USA {vikasy, bethard, msurdeanu}@email.arizona.edu Abstract Evidence retrieval is a critical stage of ques- tion answering (QA), necessary not only to im- prove performance, but also to explain the de- cisions of the corresponding QA method. We introduce a simple, fast, and unsupervised it- erative evidence retrieval method, which relies on three ideas: (a) an unsupervised alignment approach to soft-align questions and answers with justification sentences using only GloVe embeddings, (b) an iterative process that re- formulates queries focusing on terms that are not covered by existing justifications, which (c) a stopping criterion that terminates retrieval when the terms in the given question and candi- date answers are covered by the retrieved justi- fications. Despite its simplicity, our approach outperforms all the previous methods (includ- ing supervised methods) on the evidence selec- tion task on two datasets: MultiRC and QASC. When these evidence sentences are fed into a RoBERTa answer classification component, we achieve state-of-the-art QA performance on these two datasets. 1 Introduction Explainability in machine learning (ML) remains a critical unsolved challenge that slows the adoption of ML in real-world applications (Biran and Cot- ton, 2017; Gilpin et al., 2018; Alvarez-Melis and Jaakkola, 2017; Arras et al., 2017). Question answering (QA) is one of the challeng- ing natural language processing (NLP) tasks that benefits from explainability. In particular, multi- hop QA requires the aggregation of multiple ev- idence facts in order to answer complex natural language questions (Yang et al., 2018). Several multi-hop QA datasets have been proposed recently (Yang et al., 2018; Khashabi et al., 2018a; Welbl et al., 2018; Dua et al., 2019; Chen and Durrett, 2019; Khot et al., 2019a; Sun et al., 2019b; Jansen and Ustalov, 2019; Rajpurkar et al., 2018). While several neural methods have achieved state-of-the- art results on these datasets (Devlin et al., 2019; Liu et al., 2019; Yang et al., 2019), we argue that many of these directions lack a human-understandable explanation of their inference process, which is necessary to transition these approaches into real- world applications. This is especially critical for multi-hop, multiple choice QA (MCQA) where: (a) the answer text may not come from an actual knowledge base passage, and (b) reasoning is re- quired to link the candidate answers to the given question (Yadav et al., 2019b). Figure 1 shows one such multi-hop example from a MCQA dataset. In this paper we introduce a simple alignment- based iterative retriever (AIR)1, which retrieves high-quality evidence sentences from unstructured knowledge bases. We demonstrate that these evi- dence sentences are useful not only to explain the required reasoning steps that answer a question, but they also considerably improve the performance of the QA system itself. Unlike several previous works that depend on supervised methods for the retrieval …

Content truncated to 500 words