Neural Data-to-Text Generation with LM-based Text Augmentation

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 758–768 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 758 Neural Data-to-Text Generation with LM-based Text Augmentation †Ernie Chang, ⊙ Xiaoyu Shen∗, †Dawei Zhu, †Vera Demberg, ⊗ Hui Su †Dept. of Language Science and Technology, Saarland University ⊙ Amazon Alexa AI, Berlin ⊗ Pattern Recognition Center, Wechat AI, Tencent Inc, China {cychang}@coli.uni-saarland.de Abstract For many new application domains for data- to-text generation, the main obstacle in train- ing neural models consists of a lack of training data. While usually large numbers of instances are available on the data side, often only very few text samples are available. To address this problem, we here propose a novel few- shot approach for this setting. Our approach automatically augments the data available for training by (i) generating new text samples based on replacing specific values by alterna- tive ones from the same category, (ii) generat- ing new text samples based on GPT-2, and (iii) proposing an automatic method for pairing the new text samples with data samples. As the text augmentation can introduce noise to the training data, we use cycle consistency as an objective, in order to make sure that a given data sample can be correctly reconstructed af- ter having been formulated as text (and that text samples can be reconstructed from data). On both the E2E and WebNLG benchmarks, we show that this weakly supervised training paradigm is able to outperform fully super- vised seq2seq models with less than 10% an- notations. By utilizing all annotated data, our model can boost the performance of a standard seq2seq model by over 5 BLEU points, estab- lishing a new state-of-the-art on both datasets. 1 Introduction Neural data-to-text generation has been the subject of much recent research. The task aims at trans- forming source-side structured data into target-side natural language text (Reiter and Dale, 2000; Barzi- lay and Lapata, 2005). While neural end-to-end systems afford the advantage of easy adaptabil- ity (Lebret et al., 2016; Wiseman et al., 2017), huge amounts of data-text pairs are still necessary to perform on par with their rule-based counterparts ∗Work done prior to joining Amazon. The Blue Spice is a restaurant that serves English cuisine. 12 <Name> The Eagle <eattype> restaurant <food> Chinese <Name> Blue Spice <eattype> restaurant <food> English The Blue Spice is a restaurant that serves English cuisine. Table + Reference Table only U L Figure 1: Few-shot scenario: The model is expected to learn data-to-text generation with few labeled instances (i.e. table-text pairs). The example is taken from the E2E dataset. (van der Lee et al., 2018). This makes using neural systems less appealing: oftentimes, in-domain text samples are not readily available, and there is a high cost to collecting in-domain texts which fit the data samples, and annotating these texts with the data labels – the cost for collecting this data might hence even outweigh the efforts of designing a rule-based system (Gkatzia, 2016). The …

Content truncated to 500 words