TaCube: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Data

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2278 - 2291 December 7-11, 2022 ©2022 Association for Computational Linguistics TACUBE: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Data Fan Zhou1∗, Mengkang Hu2, Haoyu Dong3†, Zhoujun Cheng1, Fan Cheng1, Shi Han3, Dongmei Zhang3 1 Shanghai Jiao Tong University, 2Harbin Institute of Technology 3Microsoft Research {zhoufan98,blankcheng,chengfan}@sjtu.edu.cn, 1190200505@stu.hit.edu.cn, {hadong, shihan, dongmeiz}@microsoft.com Abstract Existing auto-regressive pre-trained language models (PLMs) like T5 and BART, have been well applied to table question answering by UNIFIEDSKG and TAPEX, respectively, and demonstrated state-of-the-art results on mul- tiple benchmarks. However, auto-regressive PLMs are challenged by recent emerging nu- merical reasoning datasets, such as TAT-QA, due to the error-prone implicit calculation. In this paper, we present TACUBE, to pre- compute aggregation/arithmetic results for the table in advance, so that they are handy and readily available for PLMs to answer numeri- cal reasoning questions. TACUBE systemat- ically and comprehensively covers a collec- tion of computational operations over table seg- ments. By simply concatenating TACUBE to the input sequence of PLMs, it shows signifi- cant experimental effectiveness. TACUBE pro- motes the F1 score from 49.6% to 66.2% on TAT-QA and achieves new state-of-the-art re- sults on WikiTQ (59.6% denotation accuracy). TACUBE ’s improvements on numerical reason- ing cases are even more notable: on TAT-QA, TACUBE promotes the exact match accuracy of BART-large by 39.6% on sum, 52.5% on average, 36.6% on subtraction and 22.2% on division. We believe that TACUBE is a general and portable pre-computation solution that can be potentially integrated into various numerical reasoning frameworks. Data and code will be available at https://github.com/ microsoft/TaCube. 1 Introduction There are numerous recent works on table-text joint reasoning, e.g., answering questions over ta- bles (Yu et al., 2018; Pasupat and Liang, 2015; Zhong et al., 2017). Meanwhile, pre-trained lan- guage models (PLMs), which have demonstrated ∗∗Work done during internship of Fan and Mengkang at Microsoft Research Asia. ††Corresponding author. Account type Pre-trained Language Models Time TACUBE Generation 2019 2018 2017 Beginning balance $(1.3) $(1.9) $(2.2) Bad debt expense (1.6) (0.6) (0.8) Write-offs, net of recoveries 1.6 1.2 1.1 Ending balance $(1.3) $(1.3) $(1.9) Cube linearization [CUBE] difference write-offs … 2019 2018 1.6 1.2 [ANSWER] : 0.4 | difference … [ANSWER] : -0.4 | … OP: diff Row header: Write-offs, … Column header: 2019, 2018 Operand: 1.6, 1.2 Result: 0.4 w/o TACUBE output: 1.7 w. TACUBE output: 0.4 Cube Input phase Answer Decoding Textual Sequence Tabular Sequence Cube Sequence  ✓ Ground Truth Answer: 0.4 implicitly calculate leverage cube [HEAD] | 2019 | 2018 | 2017 [ROW] 1 | Beginning … [ROW] 2 | Bad debt expense… What was the change in Write-offs, net of recoveries between 2018 and 2019? Table linearization Figure 1: Augmenting auto-regressive PLMs with TACUBE. By simply concatenating TACUBE to the input sequence, TACUBE significantly mitigates the cal- culation challenge in numerical reasoning over tables. great success on various natural language (NL) tasks, are also well applied to table-text joint rea- soning and …

Content truncated to 500 words