Few-Shot Table Understanding: A Benchmark Dataset and Pre-Training Baseline

Proceedings of the 29th International Conference on Computational Linguistics, pages 3741–3752 October 12–17, 2022. 3741 Few-Shot Table Understanding: A Benchmark Dataset and Pre-Training Baseline Ruixue Liu∗, Shaozu Yuan∗, Aijun Dai, Lei Shen Tiangang Zhu, Meng Chen, Xiaodong He JD AI, Beijing, China {liuruixue, yuanshaozu, daiaijun1, shenlei20}@jd.com {zhutiangang3, chenmeng20, xiaodong.he}@jd.com Abstract Few-shot table understanding is a critical and challenging problem in real-world scenario as annotations over large amount of tables are usually costly. Pre-trained language models (PLMs), which have recently flourished on tab- ular data, have demonstrated their effectiveness for table understanding tasks. However, few- shot table understanding is rarely explored due to the deficiency of public table pre-training corpus and well-defined downstream bench- mark tasks, especially in Chinese. In this paper, we establish a benchmark dataset, FewTUD, which consists of 5 different tasks with human annotations to systematically explore the few- shot table understanding in depth. Since there is no large number of public Chinese tables, we also collect a large-scale, multi-domain tab- ular corpus to facilitate future Chinese table pre-training, which includes one million tables and related natural language text with auxil- iary supervised interaction signals. Finally, we present FewTPT, a novel table PLM with rich interactions over tabular data, and evaluate its performance comprehensively on the bench- mark. Our dataset and model will be released to the public soon. 1 Introduction Relational tables, as a typical form of structured data on the Web, store a vast amount of knowledge. Table understanding (Wang et al., 2012) aims to understand the semantics of tabular data as well as the associated text jointly, which further improves the evaluation results of several tasks, including table question answering (Khalid et al., 2007; Sun et al., 2016; Bogin et al., 2019), table retrieval (Zhang and Balog, 2018), and table fact verifica- tion (Chen et al., 2020; Zhang et al., 2020). Re- cently, inspired by the huge success of pre-trained language models (PLMs) in understanding free- form natural language (NL) sentences, researchers *Both authors contributed equally to this work. have attempted to model structured data using pre- training techniques. Various table pre-training mod- els (Herzig et al., 2020; Yin et al., 2020; Yu et al., 2021; Liu et al., 2021; Cheng et al., 2021; Shi et al., 2022; Dong et al., 2022) have been proposed and made remarkable progress in learning the struc- tured schema of tables and the alignment between the input text and the schema. In real-world scenario, table understanding usu- ally faces more challenging situations, in which tables are from different domains and each table contains very limited annotations. Despite its im- portance, few-shot (Lake et al., 2015) table under- standing is rarely explored in previous works due to the following three obstacles. First, the defi- ciency of public well-designed benchmark datasets for few-shot table understanding makes the model evaluation inconvenient. Second, the lack of public large-scale high-quality table pre-training corpora blocks the exploration of table PLMs. Lastly, a table pre-training baseline tailored to the few-shot table understanding is also needed for …

Content truncated to 500 words