Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity

Proceedings of the 28th International Conference on Computational Linguistics, pages 2410–2424 Barcelona, Spain (Online), December 8-13, 2020 2410 Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity Hamza Harkous Amazon Alexa hamza.harkous@gmail.com Isabel Groves Amazon Alexa isabeg@amazon.com Amir Saffari Amazon Alexa amsafari@amazon.com Abstract End-to-end neural data-to-text (D2T) generation has recently emerged as an alternative to pipeline-based architectures. However, it has faced challenges generalizing to new domains and generating semantically consistent text. In this work, we present DATATUNER, a neural, end-to-end data-to-text generation system that makes minimal assumptions about the data repre- sentation and target domain. We take a two-stage generation-reranking approach, combining a fine-tuned language model with a semantic fidelity classifier. Each component is learnt end-to- end without needing dataset-specific heuristics, entity delexicalization, or post-processing. We show that DATATUNER achieves state of the art results on automated metrics across four major D2T datasets (LDC2017T10, WebNLG, ViGGO, and Cleaned E2E), with fluency assessed by human annotators as nearing or exceeding the human-written reference texts. Our generated text has better semantic fidelity than the state of the art on these datasets. We further demonstrate that our model-based semantic fidelity scorer is a better assessment tool compared to traditional heuristic-based measures of semantic accuracy. 1 Introduction Data-to-Text generation (D2T) is defined as automatically generating natural language texts from non- linguistic inputs (Reiter and Dale, 2000). Interest in this task has been driven by its applicability to specialized domains. For instance, D2T has been applied to generating weather reports (Liang et al., 2009), restaurant descriptions (Novikova et al., 2017b), and video game dialogues (Juraska et al., 2019). Recently, researchers have investigated D2T with more diverse domains to arrive at more generalizable text generation (such as works on LDC2017T10 (Knight et al., 2017) and WebNLG (Gardent et al., 2017) datasets). Traditional approaches to D2T follow a pipeline-based methodology, dividing the problem into several sub-problems (Reiter and Dale, 2000; Gatt and Krahmer, 2018). These include content selection (which information to include in the text), text structuring (the order in which to present the data), sentence aggregation (which information goes in individual sentences), lexicalization (finding the right words and phrases to express the data), referring expression generation (selecting the words and phrases to identify domain objects), and linguistic realization (combining all the generated words and phrases into well- formed sentences). In recent years, there has been a growing interest in going beyond pipeline-based approaches to- wards end-to-end (E-to-E) methods driven by recent advancements in deep learning (Lebret et al., 2016; Novikova et al., 2017b; Castro Ferreira et al., 2019; Duˇsek et al., 2020). Such methods can be trained with (data,text) tuples that can be efficiently collected at scale. In contrast, each step in pipeline-based approaches requires its own setup and training data, such as semantic alignments between sections of text and components of the meaning representation (MR). This makes them more costly and complex to develop and more prone to error propagation. This work is licensed under a Creative Commons Attribution 4.0 …

Content truncated to 500 words