Answering Ambiguous Questions via Iterative Prompting

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7669–7683 July 9-14, 2023 ©2023 Association for Computational Linguistics Answering Ambiguous Questions via Iterative Prompting Weiwei Sun1 Hengyi Cai2 Hongshen Chen2 Pengjie Ren1 Zhumin Chen1 Maarten de Rijke3 Zhaochun Ren1∗ 1Shandong University, Qingdao, China 2JD.com, Beijing, China 3University of Amsterdam, Amsterdam, The Netherlands {sunnweiwei,hengyi1995}@gmail.com ac@chenhongshen.com,m.derijke@uva.nl {renpengjie,chenzhumin,zhaochun.ren}@sdu.edu.cn Abstract In open-domain question answering, due to the ambiguity of questions, multiple plausible an- swers may exist. To provide feasible answers to an ambiguous question, one approach is to directly predict all valid answers, but this can struggle with balancing relevance and di- versity. An alternative is to gather candidate answers and aggregate them, but this method can be computationally costly and may neglect dependencies among answers. In this paper, we present AmbigPrompt to address the imper- fections of existing approaches to answering ambiguous questions. Specifically, we inte- grate an answering model with a prompting model in an iterative manner. The prompt- ing model adaptively tracks the reading pro- cess and progressively triggers the answering model to compose distinct and relevant an- swers. Additionally, we develop a task-specific post-pretraining approach for both the answer- ing model and the prompting model, which greatly improves the performance of our frame- work. Empirical studies on two commonly- used open benchmarks show that Ambig- Prompt achieves state-of-the-art or competitive results while using less memory and having a lower inference latency than competing ap- proaches. Additionally, AmbigPrompt also per- forms well in low-resource settings. The code are available at: https://github.com/ sunnweiwei/AmbigPrompt. 1 Introduction Recent years have witnessed substantial advances in open-domain question answering (QA) sys- tems (Karpukhin et al., 2021; Lewis et al., 2020; Izacard and Grave, 2021b), which aim to find the answer for the given question from a large knowl- edge corpus (Chen et al., 2017). While a dominat- ing scenario is the single-answer QA setting, i.e., only one exact answer is required for a given ques- tion (Karpukhin et al., 2021), this work focuses ∗Corresponding author. Which movie was both directed and screen written by Kamal Haasan? Vishwaroopam (titled vishwaroop in hindi;) is a 2013 Indian espionage action thriller film written, directed and produced by kamal haasan, who also enacts the lead role. Vishwaroopam II, or vishwaroop II, is a 2018 indian espionage action thriller film written and directed by kamal haasan, it is the sequel to “vishwaroopam” (2013) and features himself alongside rahul bose, shekhar kapur, pooja kumar and andrea jeremiah, reprising their roles Question Passages Answers Vishwaroopam Vishwaroopam II Sabaash Naidu Virumaandi Figure 1: An example of an open-domain question, a subset of its evidential Wikipedia passages and multiple answers they lead to. on the more realistic scenario of Multi-answer QA, where multiple plausible answers are associated with a user-issued question (Min et al., 2020), given that questions posed by humans are often open- ended and ambiguous.1 A natural approach for answering ambiguous open-domain questions would be to fine-tune a pre- trained answer generation model, e.g., T5 …

Content truncated to 500 words