Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4635–4641 June 6–11, 2021. ©2021 Association for Computational Linguistics 4635 Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval Chen Zhao University of Maryland chenz@cs.umd.edu Chenyan Xiong Microsoft Research chenyan.xiong@microsoft.com Jordan Boyd-Graber University of Maryland jbg@umiacs.umd.edu Hal Daum´e III Microsoft Research & University of Maryland me@hal3.name Abstract Complex question answering often requires finding a reasoning chain that consists of mul- tiple evidence pieces. Current approaches in- corporate the strengths of structured knowl- edge and unstructured text, assuming text cor- pora is semi-structured. Building on dense re- trieval methods, we propose a new multi-step retrieval approach (BEAMDR) that iteratively forms an evidence chain through beam search in dense representations. When evaluated on multi-hop question answering, BEAMDR is competitive to state-of-the-art systems, with- out using any semi-structured information. Through query composition in dense space, BEAMDR captures the implicit relationships between evidence in the reasoning chain. The code is available at https://github.com/ henryzhao5852/BeamDR. 1 Introduction Answering complex questions requires combin- ing knowledge pieces through multiple steps into an evidence chain (Ralph Hefferline → Columbia University in Figure 1). When the avail- able knowledge sources are graphs or databases, constructing chains can use the sources’ inherent structure. However, when the information needs to be pulled from unstructured text (which often has better coverage), standard information retrieval (IR) approaches only go “one hop”: from a query to a single passage. Recent approaches (Dhingra et al., 2020; Zhao et al., 2020a,b; Asai et al., 2020, inter alia) try to achieve the best of both worlds: use the un- structured text of Wikipedia with its structured hyperlinks. While they show promise on bench- marks, it’s difficult to extend them beyond aca- demic testbeds because real-world datasets often lack this structure. For example, medical records lack links between reports. Dense retrieval (Lee et al., 2019; Guu et al., 2020; Karpukhin et al., 2020, inter alia) provides a Question P1 P2 P1 Q Query Composition First Step Second Step Question: Ralph Hefferline was a psychology professor at a university that is located in what city? Evidence Chain: Ralph Hefferline -> Columbia University P1: Ralph Hefferline Ralph Franklin Hefferline was a psychology professor at Columbia University. P2: Columbia University Columbia University is a private Ivy League research university in Upper Manhattan, New York City. P P P P P P Figure 1: Top: A complex question example from HOT- POTQA that requires finding an evidence chain. Bot- tom: BEAMDR iteratively composes the new query and retrieves evidence in dense space without the need for linked documents. promising path to overcome this limitation. It en- codes the query and evidence (passage) into dense vectors and matches them in the embedding space. In addition to its efficiency—thanks to maximum inner-product search (MIPS)—Xiong et al. (2021a) show that dense retrieval rivals BERT (Devlin et al., 2019)-based (sparse) retrieve-then-rerank IR pipelines on single step retrieval. Unlike traditional term-based retrieval, fully learnable dense encod- ings …

Content truncated to 500 words