The Web as a Knowledge-Base for Answering Complex Questions

Proceedings of NAACL-HLT 2018, pages 641–651 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics The Web as a Knowledge-base for Answering Complex Questions Alon Talmor Tel-Aviv University alontalmor@mail.tau.ac.il Jonathan Berant Tel-Aviv University joberant@cs.tau.ac.il Abstract Answering complex questions is a time- consuming activity for humans that requires reasoning and integration of information. Re- cent work on reading comprehension made headway in answering simple questions, but tackling complex questions is still an ongo- ing research challenge. Conversely, seman- tic parsers have been successful at handling compositionality, but only when the informa- tion resides in a target knowledge-base. In this paper, we present a novel framework for an- swering broad and complex questions, assum- ing answering simple questions is possible us- ing a search engine and a reading comprehen- sion model. We propose to decompose com- plex questions into a sequence of simple ques- tions, and compute the final answer from the sequence of answers. To illustrate the viabil- ity of our approach, we create a new dataset of complex questions, COMPLEXWEBQUES- TIONS, and present a model that decomposes questions and interacts with the web to com- pute an answer. We empirically demonstrate that question decomposition improves perfor- mance from 20.8 precision@1 to 27.5 preci- sion@1 on this new dataset. 1 Introduction Humans often want to answer complex questions that require reasoning over multiple pieces of ev- idence, e.g., “From what country is the winner of the Australian Open women’s singles 2008?”. Answering such questions in broad domains can be quite onerous for humans, because it requires searching and integrating information from multi- ple sources. Recently, interest in question answering (QA) has surged in the context of reading comprehen- sion (RC), where an answer is sought for a ques- tion given one or more documents (Hermann et al., 2015; Joshi et al., 2017; Rajpurkar et al., 2016). q :What city is the birthplace of the author of ‘Without end’, and hosted Euro 2012? Decompose: q1 : Author of ‘Without End’? {Ken Follett, Adam Zagajewski} q2 : Birthplace of Ken Follett {Cardiff} q3 : Birthplace of Adam Zagajewski {Lviv} q4 : What cities hosted Euro 2012? {Warsaw, Kiev, Lviv, ...} Recompose: a :({Cardiff} ∪ {Lviv}) ∩ {Warsaw, Kiev, Lviv, ...}={Lviv} Figure 1: Given a complex questions q, we decom- pose the question to a sequence of simple questions q1, q2, . . . , use a search engine and a QA model to an- swer the simple questions, from which we compute the final answer a. Neural models trained over large datasets led to great progress in RC, nearing human-level perfor- mance (Wang et al., 2017). However, analysis of models revealed (Jia and Liang, 2017; Chen et al., 2016) that they mostly excel at matching questions to local contexts, but struggle with questions that require reasoning. Moreover, RC assumes docu- ments with the information relevant for the answer are available – but when questions are complex, even retrieving the documents can be difficult. Conversely, work on QA through semantic …

Content truncated to 500 words