uFACT: Unfaithful Alien-Corpora Training for Semantically Consistent Data-to-Text Generation

Findings of the Association for Computational Linguistics: ACL 2022, pages 2836 - 2841 May 22-27, 2022 c⃝2022 Association for Computational Linguistics uFACT: Unfaithful Alien-Corpora Training for Semantically Consistent Data-to-Text Generation Tisha Anders, Alexandru Coca, Bill Byrne Department of Engineering, University of Cambridge, United Kingdom anderstisha@gmail.com ac2123@cam.ac.uk wjb31@cam.ac.uk Abstract We propose UFACT (Un-Faithful Alien Corpora Training), a training corpus construc- tion method for data-to-text (d2t) generation models. We show that d2t models trained on UFACT datasets generate utterances which rep- resent the semantic content of the data sources more accurately compared to models trained on the target corpus alone. Our approach is to aug- ment the training set of a given target corpus with alien corpora which have different seman- tic representations. We show that while it is important to have faithful data from the target corpus, the faithfulness of additional corpora only plays a minor role. Consequently, UFACT datasets can be constructed with large quanti- ties of unfaithful data, minimising the need for faithful data. We show how UFACT can be leveraged to obtain state-of-the-art results on the WebNLG benchmark using METEOR as our performance metric. Furthermore, we in- vestigate the sensitivity of the generation faith- fulness to the training corpus structure using the PARENT metric, and provide a baseline for this metric on the WebNLG (Gardent et al., 2017) benchmark to facilitate comparisons with future work. 1 Introduction Data-to-text (d2t) generation is the task of gener- ating fluent text t given a set of information units, linearised into data source string d (Table 1). d {(name, Einstein), (born, 1879), (profession, physicist)} t Einstein was a physicist, born in 1879. Table 1: Example of d2t system input (d) and output (t) Training high quality generation models requires corpora whose reference texts are faithful to the data sources representing their semantic content, i.e. the reference texts tr should have perfect infor- mation overlap with d. Most corpora are, however, noisy, with imperfect fact overlap between data d and reference text tr (Dhingra et al., 2019a). The quality of the training data in that case negatively impacts the performance of a d2t generator trained on it, as well as making it difficult to estimate the true accuracy of a generation tg, given tr (Parikh et al., 2020). Faithful examples are however expen- sive to obtain, and usually only available in small quantities. In the context of this scarcity, we pro- pose the UFACT training set construction method. UFACT allows a generator to learn a more accurate d2t generation model from a mixture of faithful and unfaithful corpora, which reduces the need for vast quantities of faithful examples. For instance, our best-performing UFACT dataset contains 88692 examples, of which only 20, 000 (24.34%) exam- ples (the ones from the target corpus) are guaran- teed to be faithful. We find that our approach leads to significant improvement in PARENT (Dhingra et al., 2019b) and METEOR (Banerjee and Lavie, 2005) compared to the conventional approach of training a d2t generator on one large unfaithful corpus. …

Content truncated to 500 words