Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2335–2345 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2335 Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction Kosuke Nishida1, Kyosuke Nishida1, Masaaki Nagata2, Atsushi Otsuka1, Itsumi Saito1, Hisako Asano1, Junji Tomita1 1 NTT Media Intelligence Laboratories, NTT Corporation 2 NTT Communication Science Laboratories, NTT Corporation kosuke.nishida.ap@hco.ntt.co.jp Abstract Question answering (QA) using textual sources for purposes such as reading compre- hension (RC) has attracted much attention. This study focuses on the task of explainable multi-hop QA, which requires the system to return the answer with evidence sentences by reasoning and gathering disjoint pieces of the reference texts. It proposes the Query Focused Extractor (QFE) model for evidence extraction and uses multi-task learning with the QA model. QFE is inspired by extractive summarization models; compared with the existing method, which extracts each evidence sentence independently, it sequentially ex- tracts evidence sentences by using an RNN with an attention mechanism on the question sentence. It enables QFE to consider the de- pendency among the evidence sentences and cover important information in the question sentence. Experimental results show that QFE with a simple RC baseline model achieves a state-of-the-art evidence extraction score on HotpotQA. Although designed for RC, it also achieves a state-of-the-art evidence extraction score on FEVER, which is a recognizing textual entailment task on a large textual database. 1 Introduction Reading comprehension (RC) is a task that uses textual sources to answer any question. It has seen significant progress since the publication of nu- merous datasets such as SQuAD (Rajpurkar et al., 2016). To achieve the goal of RC, systems must be able to reason over disjoint pieces of informa- tion in the reference texts. Recently, multi-hop question answering (QA) datasets focusing on this capability, such as QAngaroo (Welbl et al., 2018) and HotpotQA (Yang et al., 2018), have been re- leased. Multi-hop QA faces two challenges. The first is the difficulty of reasoning. It is difficult for the Figure 1: Concept of explainable multi-hop QA. Given a question and multiple textual sources, the system ex- tracts evidence sentences from the sources and returns the answer and the evidence. system to find the disjoint pieces of information as evidence and reason using the multiple pieces of such evidence. The second challenge is inter- pretability. The evidence used to reason is not nec- essarily located close to the answer, so it is diffi- cult for users to verify the answer. Yang et al. (2018) released HotpotQA, an ex- plainable multi-hop QA dataset, as shown in Fig- ure 1. Hotpot QA provides the evidence sentences of the answer for supervised learning. The evi- dence extraction in multi-hop QA is more difficult than that in other QA problems because the ques- tion itself may not provide a clue for finding ev- idence sentences. As shown in Figure 1, the sys- tem finds an evidence sentence (Evidence 2) by re- lying on another evidence …

Content truncated to 500 words