Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3263–3276 August 1–6, 2021. ©2021 Association for Computational Linguistics 3263 Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction Yifan Gao†∗, Henghui Zhu‡, Patrick Ng‡, Cicero Nogueira dos Santos‡, Zhiguo Wang‡, Feng Nan‡, Dejiao Zhang‡, Ramesh Nallapati‡, Andrew O. Arnold‡, Bing Xiang‡ ‡AWS AI † The Chinese University of Hong Kong †yifangao95@gmail.com ‡{henghui,patricng,cicnog,zhiguow}@amazon.com ‡{nanfen,dejiaoz,rnallapa,anarnld,bxiang}@amazon.com Abstract In open-domain question answering, questions are highly likely to be ambiguous because users may not know the scope of relevant top- ics when formulating them. Therefore, a sys- tem needs to find possible interpretations of the question, and predict one or multiple plau- sible answers. When multiple plausible an- swers are found, the system should rewrite the question for each answer to resolve the ambi- guity. In this paper, we present a model that aggregates and combines evidence from multi- ple passages to adaptively predict a single an- swer or a set of question-answer pairs for am- biguous questions. In addition, we propose a novel round-trip prediction approach to itera- tively generate additional interpretations that our model fails to find in the first pass, and then verify and filter out the incorrect question- answer pairs to arrive at the final disam- biguated output. Our model, named REFUEL, achieves a new state-of-the-art performance on the AMBIGQA dataset, and shows com- petitive performance on NQ-OPEN and Trivi- aQA. The proposed round-trip prediction is a model-agnostic general approach for answer- ing ambiguous open-domain questions, which improves our REFUEL as well as several base- line models. We release source code for our models and experiments at https://github. com/amzn/refuel-open-domain-qa. 1 Introduction Open-domain Question Answering (QA) is the task of answering questions using a collection of pas- sages with diverse topics (Chen et al., 2017; Guu et al., 2020; Karpukhin et al., 2020). Open-domain questions are highly likely to be ambiguous be- cause people may not have the knowledge of rele- vant topics when formulating them. For example, in Figure 1, the prompt question “What’s the most ∗Work done during an internship at AWS AI. Prompt Question (Google search query): What’s the most points scored in an NBA game? Disambiguated QA Pairs: Q1: What’s the most points scored in an NBA game by combined team? / A1: 370 Q2: What’s the most points scored in an NBA game by a single team? / A2: 186 Q3: What’s the most points scored in an NBA game by an individual? / A3: 100 Relevant Wikipedia Page 1: The highest-scoring regular season game is the triple-overtime game between ... the two teams combined to score 370 points, with the pistons defeating the nuggets 186–184 ... Relevant Wikipedia Page 2: Wilt Chamberlain scored an nba-record 100 points ... Figure 1: An example from the AMBIGQA (Min et al., 2020) dataset. The Prompt Question is gathered from Google search queries and has three interpretations upon reading Wikipedia. Disambiguated QA Pairs are the …

Content truncated to 500 words