A Survey for Efficient Open Domain Question Answering

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 14447–14465 July 9-14, 2023 ©2023 Association for Computational Linguistics A Survey for Efficient Open Domain Question Answering Qin Zhang1, Shangsi Chen1, Dongkuan Xu2, Qingqing Cao3, Xiaojun Chen1, Trevor Cohn4, Meng Fang5∗ 1Shenzhen University; 2North Carolina State University; 3University of Washington; 4The University of Melbourne; 5University of Liverpool {qinzhang@, chenshangsi2021@email., xjchen@}szu.edu.cn; dxu27@ncsu.edu; qicao@cs.washington.edu; tcohn@unimelb.edu.au; Meng.Fang@liverpool.ac.uk Abstract Open domain question answering (ODQA) is a longstanding task aimed at answering factual questions from a large knowledge corpus with- out any explicit evidence in natural language processing (NLP). Recent works have predomi- nantly focused on improving the answering ac- curacy and have achieved promising progress. However, higher accuracy often requires more memory consumption and inference latency, which might not necessarily be efficient enough for direct deployment in the real world. Thus, a trade-off between accuracy, memory consump- tion and processing speed is pursued. In this paper, we will survey recent advancements in the efficiency of ODQA models and conclude core techniques for achieving efficiency. Addi- tionally, we will provide a quantitative analysis of memory cost, query speed, accuracy, and overall performance comparison. Our goal is to keep scholars informed of the latest advance- ments and open challenges in ODQA efficiency research and contribute to the further develop- ment of ODQA efficiency. 1 Introduction Open domain question answering (Voorhees and Tice, 2000) is a longstanding task in natural lan- guage processing that can answer factoid ques- tions, from a large knowledge corpus such as Wikipedia (Wikipedia, 2004) or BookCorpus (Zhu et al., 2015). Traditional QA models rely on ex- plicit evidence texts to locate the answer (Cao et al., 2019; Khashabi et al., 2020; Huang et al., 2021), while ODQA models require the processing of large amounts of knowledge quickly to answer in- put questions. And compared to search engines, ODQA models aim to enhance user-friendliness by presenting the final answer to a question directly, rather than returning a list of relevant snippets or hyperlinks (Zhu et al., 2021). ∗*Corresponding author. Recently, ODQA systems have attracted consid- erable research attention and a classic framework question encoder document encoder corpus search reader 23 June 1912 indexing （Ⅰ） retriever generator 23 June 1912 （c） candidate evidences retriever reader 23 June 1912 （a） retriever 23 June 1912 （b） What is Alan Turing’s birthday? What is Alan Turing’s birthday? What is Alan Turing’s birthday? What is Alan Turing’s birthday? Figure 1: The general pipeline of ODQA models is shown in (I), along with three different ODQA frameworks: Retriever- Reader (a), Retriever-Only (b), and Generator-Only (c). of the ODQA system is implemented by encom- passing an information retriever (IR) and a reader, i.e., Retriever-Reader (Chen et al., 2017). The task of IR is to retrieve evidence pieces from a large knowledge corpus. Popularly used IR can be TF- IDF (Chen et al., 2017), BM25 (Mao et al., 2021) and DPR (Karpukhin et al., 2020), etc. The target of the reader is understanding and …

Content truncated to 500 words