Neural Pipeline for Zero-Shot Data-to-Text Generation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3914 - 3932 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Neural Pipeline for Zero-Shot Data-to-Text Generation Zdenˇek Kasner and Ondˇrej Dušek Charles University, Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics Prague, Czech Republic {kasner,odusek}@ufal.mff.cuni.cz Abstract In data-to-text (D2T) generation, training on in-domain data leads to overfitting to the data representation and repeating training data noise. We examine how to avoid finetuning pretrained language models (PLMs) on D2T generation datasets while still taking advantage of surface realization capabilities of PLMs. Inspired by pipeline approaches, we propose to generate text by transforming single-item descriptions with a sequence of modules trained on general- domain text-based operations: ordering, aggre- gation, and paragraph compression. We train PLMs for performing these operations on a syn- thetic corpus WIKIFLUENT which we build from English Wikipedia. Our experiments on two major triple-to-text datasets—WebNLG and E2E—show that our approach enables D2T generation from RDF triples in zero-shot set- tings.1 1 Introduction The aim of data-to-text (D2T) generation is to pro- duce natural language descriptions of structured data (Gatt and Krahmer, 2018; Reiter and Dale, 1997). Although pipelines of rule-based D2T gener- ation modules are still used in practice (Dale, 2020), end-to-end approaches based on PLMs recently showed superior benchmark performance (Ke et al., 2021; Chen et al., 2020a; Ferreira et al., 2020; Kale and Rastogi, 2020b; Ribeiro et al., 2020), surpass- ing pipeline systems (Ferreira et al., 2019) in both automatic and human evaluation metrics. Finetuning PLMs on human-written references is widely accepted as a standard approach for adapt- ing PLMs to the D2T generation objective and achieving good performance on a given bench- mark (Agarwal et al., 2021; Ke et al., 2021). How- ever, finetuning a model on the domain-specific data leads to overfitting to the particular bench- mark, decreasing performance on out-of-domain 1Our code and data is available at https://github. com/kasnerz/zeroshot-d2t-pipeline. a b William Anders was a crew member of Apollo 8. wasACrewMemberOf William Anders Apollo 8 birthPlace William Anders British Hong Kong birthDate William Anders 1933-10-17 William Anders was born on 1933-10-17 in British Hong Kong. He was a crew member of Apollo 8. c William Anders was born on 1933-10-17. William Anders was born in British Hong Kong. ordering aggregation paragraph compression b a c a b + c <sep> Figure 1: A scheme of our pipeline for zero-shot data- to-text generation from RDF triples: (1) ordering, (2) aggregation, (3) paragraph compression. Individual pipeline modules are trained on a large general-domain text corpus and operate over text in natural language. In-domain knowledge is included only in the simple hand-crafted templates for each predicate. data (Laha et al., 2019). Gathering a large set of ref- erences for a particular domain is also costly and time-consuming as it usually requires collecting human-written references through crowdsourcing (Dušek et al., 2020). These problems can be par- tially mitigated using few-shot approaches (Chen et al., 2020b; Ke et …

Content truncated to 500 words