MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6322–6334 July 9-14, 2023 ©2023 Association for Computational Linguistics MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering Vaishali Pal1,2 Andrew Yates1 Evangelos Kanoulas1 Maarten de Rijke1 1University of Amsterdam, The Netherlands 2Discovery Lab, Elsevier, The Netherlands v.pal, a.c.yates, e.kanoulas, m.derijke@uva.nl Abstract Recent advances in tabular question answer- ing (QA) with large language models are con- strained in their coverage and only answer ques- tions over a single table. However, real-world queries are complex in nature, often over multi- ple tables in a relational database or web page. Single table questions do not involve common table operations such as set operations, Carte- sian products (joins), or nested queries. Fur- thermore, multi-table operations often result in a tabular output, which necessitates table gen- eration capabilities of tabular QA models. To fill this gap, we propose a new task of answer- ing questions over multiple tables. Our model, MultiTabQA, not only answers questions over multiple tables, but also generalizes to generate tabular answers. To enable effective training, we build a pre-training dataset comprising of 132,645 SQL queries and tabular answers. Fur- ther, we evaluate the generated tables by intro- ducing table-specific metrics of varying strict- ness assessing various levels of granularity of the table structure. MultiTabQA outperforms state-of-the-art single table QA models adapted to a multi-table QA setting by finetuning on three datasets: Spider, Atis and GeoQuery. 1 Introduction Question answering (QA) over multiple tables aims to provide exact answers to natural language ques- tions with evidence from one or more tables (Jin et al., 2022). This is in contrast to single-table QA, which has been the focus of tabular QA research to date (Liu et al., 2021; Nan et al., 2021; Zhu et al., 2021; Herzig et al., 2020). Even though groups of related tables are ubiquitous in real-world cor- pora, such as relational databases or tables in a web page, multi-table QA remains a largely unexplored area. To address this gap, we propose a new task of answering questions over multiple tables. Our multi-table QA model, MultiTabQA,1 addresses 1Code and data are at: https://github.com/ kolk/MultiTabQA SELECT zip_code FROM weather GROUP BY zip_code HAVING avg ( mean_humidity ) < 70 INTERSECT SELECT zip_code FROM trip GROUP BY zip_code HAVING count ( * ) >= 200 Weather cloud_cover min_humidity zip_code trip duration end_date zip_code What is the zip code of trips made which are above 200 with humidity below 70 Multi-Table QA 97010 92231 zip_code Figure 1: Multi-table QA. The QA model generates a tabular answer from either a natural language question or an SQL query and one or more tables as input context. novel challenges introduced by multi-table context. These include complex queries involving chains of reasoning, disambiguation of relevant table names at each reasoning step, and generating a final table as answer. It also leads to novel question-types that are unnatural in a single-table setting. For in- stance, questions involving operations specific to multiple …

Content truncated to 500 words