Few-shot Table-to-text Generation with Prefix-Controlled Generator

Proceedings of the 29th International Conference on Computational Linguistics, pages 6493–6504 October 12–17, 2022. 6493 Few-Shot Table-to-Text Generation with Prefix-Controlled Generator Yutao Luo , Menghua Lu , Gongshen Liu∗, Shilin Wang Shanghai Jiao Tong University {luoyt1996, 610228633, lgshen, wsl}@sjtu.edu.cn Abstract Neural table-to-text generation approaches are data-hungry, limiting their adaptation for low- resource real-world applications. Previous works mostly resort to Pre-trained Language Models (PLMs) to generate fluent summaries of a table. However, they often contain halluci- nated contents due to the uncontrolled nature of PLMs. Moreover, the topological differences between tables and sequences are rarely stud- ied. Last but not least, fine-tuning on PLMs with a handful of instances may lead to over- fitting and catastrophic forgetting. To alle- viate these problems, we propose a prompt- based approach, Prefix-Controlled Generator (i.e., PCG), for few-shot table-to-text gener- ation. We prepend a task-specific prefix for a PLM to make the table structure better fit the pre-trained input. In addition, we generate an input-specific prefix to control the factual contents and word order of the generated text. Both automatic and human evaluations on dif- ferent domains (humans, books and songs) of the Wikibio dataset show substantial improve- ments over baseline approaches. 1 Introduction Table-to-text generation is a significant branch of Natural Language Generation (NLG), aiming at generating descriptive text given an input table. There is a wide range of application scenarios for automatic table-to-text generation, such as sport news generation (Wiseman et al., 2017), story gen- eration (Liu et al., 2020), weather forecasting re- port (Liang et al., 2009), and open-domain question answering (Chen et al., 2021). Recent years have witnessed the great develop- ment of pre-trained language models (PLMs) (De- vlin et al., 2019; Radford et al., 2019; Lewis et al., 2020), which achieve state-of-the-art performance on many text generation tasks, such as neural ma- chine translation, document summarization, etc. ∗Corresponding author. Unlike these tasks, table-to-text generation faces the lack of labeled data. Due to the development of data science, many statistical tables are generated in our daily life, but they scarcely have correspond- ing natural language descriptions, which limits the real-world application of data-hungry pre-trained models. To address this problem, researchers inves- tigate workarounds in the few-shot setting. Chen et al. (2020), Gong et al. (2020) and Su et al. (2021a) leverage pre-trained linguistic knowledge of neural language models, then fine-tune them in target domains with limited labeled data. This “pre-train and fine-tune” paradigm performs rela- tively well in generating descriptive text from ta- bles. Recently, another paradigm named “pre-train and prompt” has been proposed in order to adapt PLMs to downstream tasks without fine-tuning, which is more suitable for few-shot and zero-shot scenarios. Li and Liang (2021) prepends prompt tokens to adapt table-to-text generation to sequen- tial generation task, and freezes PLMs’ weights to fully leverage their prior knowledge learned in the pre-training stage. Despite their contributions, however, two main challenges for table-to-text generation remain to be explored, namely (1) the topological structure difference between tables and sequential inputs and (2) …

Content truncated to 500 words