Generative Context Pair Selection for Multi-hop Question Answering

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7009–7015 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 7009 Generative Context Pair Selection for Multi-hop Question Answering Dheeru Dua♣ Cicero Nogueira dos Santos♠ Patrick Ng♠ Ben Athiwaratkun♠ Bing Xiang♠ Matt Gardner♥ Sameer Singh♣ ♣University of California, Irvine, USA ♠Amazon Web Services, New York ♥Allen Institute for Artificial Intelligence ddua@uci.edu Abstract Compositional reasoning tasks such as multi- hop question answering require models to learn how to make latent decisions using only weak supervision from the final an- swer. Crowdsourced datasets gathered for these tasks, however, often contain only a slice of the underlying task distribution, which can induce unanticipated biases such as shal- low word overlap between the question and context. Recent works have shown that dis- criminative training results in models that ex- ploit these underlying biases to achieve a bet- ter held-out performance, without learning the right way to reason. We propose a generative context selection model for multi-hop QA that reasons about how the given question could have been generated given a context pair and not just independent contexts. We show that on HotpotQA, our proposed generative pas- sage selection model, while being comparable to the state-of-the-art answering performance, has a better performance (4.9% higher than baseline) on an adversarial held-out set that tests the robustness of model’s multi-hop rea- soning capabilities. 1 Introduction Recently many reading comprehension datasets like HotpotQA (Yang et al., 2018) and Wiki- Hop (Welbl et al., 2018) that require compositional reasoning over several disjoint passages have been introduced. This style of compositional reason- ing, also referred to as multi-hop reasoning, first requires finding the correct set of passages relevant to the question and then finding the answer span in the selected set of passages. These dataset are often collected via crowdsourcing, which makes the training and evaluation of such models heavily reliant on the quality of the collected held-out sets. Crowdsourced datasets, however, often present only a partial picture of the underlying data dis- tribution. Learning complex latent sequential de- cisions, like in multi-hop reasoning, to answer a Question: The 2011-12 VCU Rams men’s basketball team, led by third year head coach Shaka Smart, represented the university which was founded in what year? Gold Answer: 1838 Passage 1: The 2011-12 VCU Rams men’s basketball team represented Virginia Commonwealth University during the 2011-12 NCAA Division I men’s basketball season... Passage 2: Virginia Commonwealth University (VCU) is a public research university located in Richmond, Virginia. VCU was founded in 1838 as the medical department of Hampden-Sydney College, becoming the Medical College of Virginia in 1854... Prediction: 1838 Adversarial context from Jiang and Bansal (2019): Dartmouth University is a public research university located in Richmond, Virginia. Dartmouth was founded in 1938 as the medical department of Hampden-Sydney College, becoming the Medical College of Virginia in 1854... New Prediction: 1938 Figure 1: Example from HotpotQA, showing the rea- soning chain for answering the question (in green) and an adversarial context (in pink) introduced …

Content truncated to 500 words