HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1094 - 1110 May 22-27, 2022 c⃝2022 Association for Computational Linguistics HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation Zhoujun Cheng1∗, Haoyu Dong2∗†, Zhiruo Wang3∗, Ran Jia2, Jiaqi Guo4 , Yan Gao2, Shi Han2, Jian-Guang Lou2, Dongmei Zhang2 1Shanghai Jiao Tong University, 2Microsoft Research Asia 3Carnegie Mellon University, 4Xi’an Jiaotong University blankcheng@sjtu.edu.cn, zhiruow@cs.cmu.edu jasperguo2013@stu.xjtu.edu.cn {hadong,jia.ran,yan.gao,shihan,jlou,dongmeiz}@microsoft.com Abstract Tables are often created with hierarchies, but existing works on table reasoning mainly fo- cus on flat tables and neglect hierarchical ta- bles. Hierarchical tables challenge table rea- soning by complex hierarchical indexing, as well as implicit relationships of calculation and semantics. We present a new dataset, HiTab, to study question answering (QA) and natu- ral language generation (NLG) over hierarchi- cal tables. HiTab is a cross-domain dataset constructed from a wealth of statistical reports and Wikipedia pages, and has unique charac- teristics: (1) nearly all tables are hierarchical, and (2) questions are not proposed by anno- tators from scratch, but are revised from real and meaningful sentences authored by analysts. (3) To reveal complex numerical reasoning in analysis, we provide fine-grained annotations of quantity and entity alignment. Experimen- tal results show that HiTab presents a strong challenge for existing baselines and a valu- able benchmark for future research. Target- ing hierarchical structure, we devise an effec- tive hierarchy-aware logical form for symbolic reasoning over tables. Furthermore, we lever- age entity and quantity alignment to explore partially supervised training in QA and con- ditional generation in NLG, which largely re- duces spurious predictions in QA and mean- ingless descriptions in NLG. The dataset and code are available at https://github.com/ microsoft/HiTab. 1 Introduction In recent years, there are a flurry of works on rea- soning over semi-structured tables, e.g., answering questions over tables (Yu et al., 2018; Pasupat and Liang, 2015) and generating fluent and faithful text from tables (Lebret et al., 2016; Parikh et al., 2020). ∗Equal contributions. Work done during Zhoujun and Zhiruo’s internship at Microsoft Research Asia. †Corresponding author. 1https://www.nsf.gov/statistics/2019/nsf19319/ • Teaching assistantships were most commonly reported as the primary mechanism of support for master's students (11%). Figure 1: A hierarchical table and accompanied descrip- tions in a National Science Foundation report.1 But they mainly focus on simple flat tables and ne- glect complex tables, e.g., hierarchical tables. A table is regarded as hierarchical if its header ex- hibits a multi-level structure (Lim and Ng, 1999; Chen and Cafarella, 2014; Wang et al., 2020). Hi- erarchical tables are widely used, especially in data products, statistical reports, and research papers in government, finance, and science-related domains. Hierarchical tables challenge QA and NLG due to: (1) Hierarchical indexing. Hierarchical head- ers, such as D2:G3 and A4:A25 in Figure 1, are informative and intuitive for readers, but make cell selection much more compositional than flat tables, requiring multi-level and bi-dimensional indexing. For example, to select the cell E5 (“66.6”), one needs to specify two …

Content truncated to 500 words