Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 16171–16189 July 9-14, 2023 ©2023 Association for Computational Linguistics Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma Patrick Ng, Zhiguo Wang, Bonan Min, William Wang Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang AWS AI Labs {hanboli, myshang, spilieva, jieman, patricng, zhiguow, bonanmin, wyw}@amazon.com {mckeownk, vittorca, drot, bxiang}@amazon.com Abstract We present a novel approach for structured data- to-text generation that addresses the limitations of existing methods that primarily focus on spe- cific types of structured data. Our proposed method aims to improve performance in multi- task training, zero-shot and few-shot scenarios by providing a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and mean- ing representations. We demonstrate that our proposed approach can effectively adapt to new structured forms, and can improve performance in comparison to current methods. For example, our method resulted in a 66% improvement in zero-shot BLEU scores when transferring mod- els trained on table inputs to a knowledge graph dataset. Our proposed method is an important step towards a more general data-to-text gener- ation framework. 1 1 Introduction Data-to-text generation is the task of converting structured data into natural language text that can be easily understood by humans. Previous methods for data-to-text generation have been limited to spe- cific structured forms. For example, graph neural networks (GNNs) have been used to encode knowl- edge graph input (Rik Koncel-Kedziorski and Ha- jishirzi, 2019; Ribeiro et al., 2020; Guo et al., 2020; Li et al., 2021), while table-specific encoders have been proposed for tables (Liu et al., 2017; Bao et al., 2018; Nema et al., 2018; Jain et al., 2018; Wang et al., 2022). However, these methods are not easily transferable to other structured forms, creating a barrier for scientific development and preventing models from learning across tasks. Re- cent work has attempted to address the problem of limited structured form applicability by using pretrained language models (PLMs) as a single text-to-text framework for all data structures, by 1Our code will be open-sourced at anonymous-link. linearizing the data as text sequences. As shown by Kale and Rastogi (2020); Xie et al. (2022), these methods achieve state-of-the-art performance on a wide range of data-to-text tasks. Despite the advancements made in the field, there are still unresolved questions regarding the relationship between various structured forms, par- ticularly in the context of zero-shot or few-shot set- tings, where models are required to rapidly adapt to new structured forms. This is particularly per- tinent in cases of data scarcity, when structured forms vary across different domains and there is a limited amount of data available for a specific structured form, but a single model is needed to operate on all of them. Such an example is to adapt a knowledge-graph-to-text model to a new domain with data in table format. Even when there is an abundance of …

Content truncated to 500 words