Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering

Proceedings of NAACL-HLT 2019, pages 335–344 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 335 Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering Jianmo Ni1∗, Chenguang Zhu2, Weizhu Chen3, Julian McAuley1 1 University of California, San Diego 2 Microsoft Speech and Dialogue Research Group 3 Microsoft Dynamics 365 AI {jin018,jmcauley}@ucsd.edu, {chezhu,wzchen}@microsoft.com Abstract Open-domain question answering remains a challenging task as it requires models that are capable of understanding questions and an- swers, collecting useful information, and rea- soning over evidence. Previous work typi- cally formulates this task as a reading com- prehension or entailment problem given evi- dence retrieved from search engines. How- ever, existing techniques struggle to retrieve indirectly related evidence when no directly related evidence is provided, especially for complex questions where it is hard to parse precisely what the question asks. In this pa- per we propose a retriever-reader model that learns to attend on essential terms during the question answering process. We build (1) an essential term selector which first identifies the most important words in a question, then refor- mulates the query and searches for related ev- idence; and (2) an enhanced reader that distin- guishes between essential terms and distract- ing words to predict the answer. We evaluate our model on multiple open-domain multiple- choice QA datasets, notably performing at the level of the state-of-the-art on the AI2 Reason- ing Challenge (ARC) dataset. 1 Introduction Open-domain question answering (QA) has been extensively studied in recent years. Many exist- ing works have followed the ‘search-and-answer’ strategy and achieved strong performance (Chen et al., 2017; Kwon et al., 2018; Wang et al., 2018b) spanning multiple QA datasets such as TriviaQA (Joshi et al., 2017), SQuAD (Rajpurkar et al., 2016), MS-Macro (Nguyen et al., 2016), ARC (Clark et al., 2018) among others. However, open-domain QA tasks become inher- ently more difficult when (1) dealing with ques- tions with little available evidence; (2) solving ∗Most of the work was done during internship at Mi- crosoft, Redmond. questions where the answer type is free-form text (e.g. multiple-choice) rather than a span among existing passages (i.e., ‘answer span’); or when (3) the need arises to understand long and com- plex questions and reason over multiple passages, rather than simple text matching. As a result, it is essential to incorporate commonsense knowledge or to improve retrieval capability to better capture partially related evidence (Chen et al., 2017). As shown in Table 1, the TriviaQA, SQuAD, and MS-Macro datasets all provide passages within which the correct answer is guaranteed to exist. However, this assumption ignores the diffi- culty of retrieving question-related evidence from a large volume of open-domain resources, espe- cially when considering complex questions which require reasoning or commonsense knowledge. On the other hand, ARC does not provide passages known to contain the correct answer. Instead, the task of identifying relevant passages is left to the solver. However, questions in ARC have multi- ple answer choices that provide indirect informa- …

Content truncated to 500 words