Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4430–4441 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4430 Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index Minjoon Seo1,5∗ Jinhyuk Lee6∗ Tom Kwiatkowski2, Ankur P. Parikh2 Ali Farhadi1,3,4 Hannaneh Hajishirzi1,3 University of Washington1 Google2 Allen Institute for AI3 XNOR.AI4 NAVER5 Korea University6 {minjoon,ali,hannaneh}@cs.washington.edu {tomkwiat,aparikh}@google.com jinhyuk lee@korea.ac.kr Abstract Existing open-domain question answering (QA) models are not suitable for real-time us- age because they need to process several long documents on-demand for every input query, which is computationally prohibitive. In this paper, we introduce query-agnostic indexable representations of document phrases that can drastically speed up open-domain QA. In par- ticular, our dense-sparse phrase encoding ef- fectively captures syntactic, semantic, and lex- ical information of the phrases and eliminates the pipeline filtering of context documents. Leveraging strategies for optimizing training and inference time, our model can be trained and deployed even in a single 4-GPU server. Moreover, by representing phrases as pointers to their start and end tokens, our model in- dexes phrases in the entire English Wikipedia (up to 60 billion phrases) using under 2TB. Our experiments on SQuAD-Open show that our model is on par with or more accurate than previous models with 6000x reduced com- putational cost, which translates into at least 68x faster end-to-end inference benchmark on CPUs. Code and demo are available at nlp. cs.washington.edu/denspi 1 Introduction Extractive open-domain question answering (QA) is usually referred to the task of answering an arbi- trary factoid question (such as “Where was Barack Obama born?”) from a general web text (such as Wikipedia). This is an extension of the read- ing comprehension task (Rajpurkar et al., 2016) of selecting an answer phrase to a question given an evidence document. To make a scalable open- domain QA system, One can leverage a search en- gine to filter the web-scale evidence to a few doc- uments, in which the answer span can be extracted using a reading comprehension model (Chen et al., ∗Equal contribution. 2017). However, the accuracy of the final QA sys- tem is bounded by the performance of the search engine due to the pipeline nature of the search process. What is more, running a neural reading comprehension model (Seo et al., 2017) on a few documents is still computationally costly, since it needs to process the evidence document for ev- ery new question at inference time. This often requires multi-GPU-seconds or tens to hundreds of CPU-seconds – BERT (Devlin et al., 2019) can process only a few thousand words per second on an Nvidia V100 GPU. In this paper, we introduce Dense-Sparse Phrase Index (DENSPI), an indexable query-agnostic phrase representation model for real-time open- domain QA. The phrase representations are in- dexed offline using efficient training and memory- efficient strategies for storage. During inference time, the input question is mapped to the same rep- resentation space, and the phrase with maximum inner product search is retrieved. Our phrase encoding model combines …

Content truncated to 500 words