Data-to-text Generation with Variational Sequential Planning

Data-to-text Generation with Variational Sequential Planning Ratish Puduppully and Yao Fu and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB, UK r.puduppully@sms.ed.ac.uk yao.fu@ed.ac.uk mlap@inf.ed.ac.uk Abstract We consider the task of data-to-text genera- tion, which aims to create textual output from non-linguistic input. We focus on generating long-form text, that is, documents with mul- tiple paragraphs, and propose a neural model enhanced with a planning component respon- sible for organizing high-level information in a coherent and meaningful way. We infer latent plans sequentially with a structured vari- ational model, while interleaving the steps of planning and generation. Text is generated by conditioning on previous variational decisions and previously generated text. Experiments on two data-to-text benchmarks (ROTOWIRE and MLB) show that our model outperforms strong baselines and is sample-efficient in the face of limited training data (e.g., a few hundred instances). 1 Introduction Data-to-text generation refers to the task of gener- ating textual output from non-linguistic input such as database tables, spreadsheets, or simulations of physical systems (Reiter and Dale, 1997, 2000; Gatt and Krahmer, 2018). Recent progress in this area (Mei et al., 2016; Lebret et al., 2016; Wiseman et al., 2017) has been greatly facil- itated by the very successful encoder-decoder neural architecture (Sutskever et al., 2014) and the development of large scale datasets. ROTOWIRE (Wiseman et al., 2017) and MLB (Puduppully et al., 2019b) constitute such examples. They both focus on the sports domain, which has histori- cally drawn attention in the generation community (Barzilay and Lapata, 2005; Tanaka-Ishii et al., 1998; Robin, 1994) and consider the problem of generating long target texts from database records. Figure 1 (reproduced from Puduppully and Lapata, 2021) provides a sample from the MLB dataset, which pairs human written summaries (Table C in Figure 1) with major league baseball game statistics. These are mostly scores (collec- tively referred to as box score) which summarize the performance of teams and players, for ex- ample, batters, pitchers, or fielders (Table A in Figure 1) and a play-by-play description of the most important events in the game (Table B in Figure 1). Game summaries in MLB are relatively long (540 tokens on average) with multiple para- graphs (15 on average). The complexity of the input and the length of the game summaries pose various challenges to neural models which, de- spite producing fluent output, are often imprecise, prone to hallucinations, and display poor con- tent selection (Wiseman et al., 2017). Attempts to address these issues have seen the development of special-purpose modules that keep track of salient entities (Iso et al., 2019; Puduppully et al., 2019b), determine which records (see the rows in Tables A and B in Figure 1) should be mentioned in a sentence and in which order (Puduppully et al., 2019a; Narayan et al., 2020), and reconceptualize the input in terms of paragraph plans (Puduppully and Lapata, 2021) to facilitate document-level planning (see Table D in Figure 1). Specifically, Puduppully and …

Content truncated to 500 words