Coarse-to-Fine Question Answering for Long Documents

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 209–220 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1020 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 209–220 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1020 Coarse-to-Fine Question Answering for Long Documents Eunsol Choi† University of Washington eunsol@cs.washington.edu Daniel Hewlett, Jakob Uszkoreit Google {dhewlett,usz}@google.com Illia Polosukhin† XIX.ai i@xix.ai Alexandre Lacoste† Element AI allac@elementai.com Jonathan Berant† Tel Aviv University joberant@cs.tau.ac.il Abstract We present a framework for question answering that can efficiently scale to longer documents while maintaining or even improving performance of state-of- the-art models. While most successful ap- proaches for reading comprehension rely on recurrent neural networks (RNNs), run- ning them over long documents is pro- hibitively slow because it is difficult to parallelize over sequences. Inspired by how people first skim the document, iden- tify relevant parts, and carefully read these parts to produce an answer, we combine a coarse, fast model for selecting rele- vant sentences and a more expensive RNN for producing the answer from those sen- tences. We treat sentence selection as a la- tent variable trained jointly from the an- swer only using reinforcement learning. Experiments demonstrate the state of the art performance on a challenging subset of the WIKIREADING dataset (Hewlett et al., 2016) and on a new dataset, while speed- ing up the model by 3.5x-6.7x. 1 Introduction Reading a document and answering questions about its content are among the hallmarks of nat- ural language understanding. Recently, interest in question answering (QA) from unstructured doc- uments has increased along with the availability of large scale datasets for reading comprehension (Hermann et al., 2015; Hill et al., 2015; Rajpurkar et al., 2016; Onishi et al., 2016; Nguyen et al., 2016; Trischler et al., 2016a). Current state-of-the-art approaches for QA over documents are based on recurrent neural networks †Work done while the authors were at Google. Query (x) Document (d) Answer (y) Sentence Selection (Latent) Answer Generation (RNN) Document Summary ( ˆd) Figure 1: Hierarchical question answering: the model first selects relevant sentences that produce a document summary ( ˆd) for the given query (x), and then generates an answer (y) based on the summary ( ˆd) and the query x. (RNNs) that encode the document and the ques- tion to determine the answer (Hermann et al., 2015; Chen et al., 2016; Kumar et al., 2016; Kadlec et al., 2016; Xiong et al., 2016). While such models have access to all the relevant infor- mation, they are slow because the model needs to be run sequentially over possibly thousands of to- kens, and the computation is not parallelizable. In fact, such models usually truncate the docu- ments and consider only a limited number of to- kens (Miller et al., 2016; Hewlett et al., 2016). Inspired by studies on how people answer ques- tions by first skimming the document, identifying relevant parts, and carefully reading these …

Content truncated to 500 words