MultiSpanQA: A Dataset for Multi-Span Question Answering

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1250 - 1260 July 10-15, 2022 ©2022 Association for Computational Linguistics MultiSpanQA: A Dataset for Multi-Span Question Answering Haonan Li♠ Maria Vasardani♦ Martin Tomko♥ Timothy Baldwin♠,♣ ♠ School of Computing and Information Systems, The University of Melbourne ♥ Department of Infrastructure Engineering, The University of Melbourne ♦ Department of Geospatial Science, RMIT University ♣ Department of Natural Language Processing, MBZUAI haonanl5@student.unimelb.edu.au, maria.vasardani2@rmit.edu.au tomkom@unimelb.edu.au, tb@ldwin.net Abstract Most existing reading comprehension datasets focus on single-span answers, which can be extracted as a single contiguous span from a given text passage. Multi-span questions, i.e., questions whose answer is a series of multiple discontiguous spans in the text, are common in real life but are less studied. In this paper, we present MultiSpanQA1, a new dataset that focuses on questions with multi-span answers. Raw questions and contexts are extracted from the Natural Questions (Kwiatkowski et al., 2019) dataset. After multi-span re-annotation, MultiSpanQA consists of over a total of 6,000 multi-span questions in the basic version, and over 19,000 examples with unanswerable ques- tions, and questions with single-, and multi- span answers in the expanded version. We in- troduce new metrics for the purposes of multi- span question answering evaluation, and estab- lish several baselines using advanced models. Finally, we propose a new model which beats all baselines and achieves the state-of-the-art on our dataset. 1 Introduction The task of reading comprehension, where models are required to process a text and answer questions about it, has seen rapid progress in recent years. As systems have increasingly matched humans on popular datasets (Rajpurkar et al., 2016, 2018), re- searchers have developed newer, more complex formulations of the task, such as very long contexts and answers (Kwiatkowski et al., 2019), multi-hop reasoning (Yang et al., 2018), and discrete oper- ations over the content of paragraphs (Dua et al., 2019). One thing these datasets have in common is that the answer is constrained to be a single span that can be extracted or computed from the context. However, in practice, the answer to a question will often consist of multiple parts. As in the exam- ple in Figure 1, the answer set contains 10 countries, 1Available at: https://multi-span.github.io Question: Which countries does the Danube River flow through? Passage: ... Originating in Germany, the Danube flows southeast for 2,850 km (1,770 mi), passing through or bordering Austria, Slovakia, Hungary, Croatia, Serbia, Romania, Bulgaria, Moldova and Ukraine before draining into the Black Sea. ... Answer set: {Germany, Austria, Slovakia, Hungary, Croatia, Serbia, Romania, Bulgaria, Moldova, Ukraine } Figure 1: Example of a multi-span question and answer pair. some of which are discontiguous in the passage. Such cases are largely ignored in existing reading comprehension research, in part because there are no datasets of multi-span questions. In this paper, we introduce MultiSpanQA, a new reading comprehension dataset consisting of 6,536 multi-span examples. The raw questions and pas- sages are extracted from …

Content truncated to 500 words