If You Want to Go Far Go Together: Unsupervised Joint Candidate Evidence Retrieval for Multi-hop Question Answering

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4571–4581 June 6–11, 2021. ©2021 Association for Computational Linguistics 4571 If You Want to Go Far Go Together: Unsupervised Joint Candidate Evidence Retrieval for Multi-hop Question Answering Vikas Yadav, Steven Bethard, Mihai Surdeanu University of Arizona, Tucson, AZ, USA {vikasy, bethard, msurdeanu}@email.arizona.edu Abstract Multi-hop reasoning requires aggregation and inference from multiple facts. To retrieve such facts, we propose a simple approach that retrieves and reranks set of evidence facts jointly. Our approach first generates unsuper- vised clusters of sentences as candidate evi- dence by accounting links between sentences and coverage with the given query. Then, a RoBERTa-based reranker is trained to bring the most representative evidence cluster to the top. We specifically emphasize on the impor- tance of retrieving evidence jointly by showing several comparative analyses to other meth- ods that retrieve and rerank evidence sen- tences individually. First, we introduce sev- eral attention- and embedding-based analy- ses, which indicate that jointly retrieving and reranking approaches can learn compositional knowledge required for multi-hop reasoning. Second, our experiments show that jointly re- trieving candidate evidence leads to substan- tially higher evidence retrieval performance when fed to the same supervised reranker. In particular, our joint retrieval and then rerank- ing approach achieves new state-of-the-art evi- dence retrieval performance on two multi-hop question answering (QA) datasets: 30.5 Re- call@2 on QASC, and 67.6% F1 on MultiRC. When the evidence text from our joint retrieval approach is fed to a RoBERTa-based answer selection classifier, we achieve new state-of- the-art QA performance on MultiRC and sec- ond best result on QASC. 1 Introduction Recent advances in question answering (QA) have achieved excellent performance on several bench- mark datasets (Wang et al., 2019a), even when rely- ing on partial (Gururangan et al., 2018), incorrect (Jia and Liang, 2017) or no supporting knowledge (Raffel et al., 2019). Specifically, black-box neural QA methods have shown to rely on spurious signals confirming unfaithful or non-explainable behavior Question: RNA is a small molecule that can squeeze through pores in (A) dermal & vascular tissue (B) space between (C) eukaryotic cells (D) jellyfish (E) · · · · · · (H) Gold evidence sentences: 1. RNA is a small molecule that can squeeze through pores in the nuclear membrane 2. Cells with a nuclear membrane are called eukaryotic. BM25 sentences: 1. RNA is a small molecule that can squeeze through pores in the nuclear membrane. 2. RNA synthesis in eukaryotic cells is synthesized by three types of RNA polymerases 3. Eukaryotic cells have three different RNA polymerases. 4. the molecule seems to have evolved specifically to para- sitize eukaryotic cells WAIR Step-1 sentences: 1. RNA is a small molecule that can squeeze through pores in the nuclear membrane. 2. RNA synthesis in eukaryotic cells is synthesized by three types of RNA polymerases WAIR Step-2 sentences: 1. Cells with a nuclear membrane are called eukaryotic 2. Eukaryotic cells have three different RNA …

Content truncated to 500 words