Curate and Generate: A Corpus and Method for Joint Control of Semantics and Style in Neural NLG

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5938–5951 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5938 Curate and Generate: A Corpus and Method for Joint Control of Semantics and Style in Neural NLG Shereen Oraby, Vrindavan Harrison, Abteen Ebrahimi, and Marilyn Walker Natural Language and Dialog Systems Lab University of California, Santa Cruz {soraby,vharriso,aaebrahi,mawalker}@ucsc.edu Abstract Neural natural language generation (NNLG) from structured meaning representations has become increasingly popular in recent years. While we have seen progress with generating syntactically correct utterances that preserve semantics, various shortcomings of NNLG sys- tems are clear: new tasks require new train- ing data which is not available or straightfor- ward to acquire, and model outputs are sim- ple and may be dull and repetitive. This pa- per addresses these two critical challenges in NNLG by: (1) scalably (and at no cost) creat- ing training datasets of parallel meaning rep- resentations and reference texts with rich style markup by using data from freely available and naturally descriptive user reviews, and (2) systematically exploring how the style markup enables joint control of semantic and stylistic aspects of neural model output. We present YELPNLG, a corpus of 300,000 rich, paral- lel meaning representations and highly stylis- tically varied reference texts spanning differ- ent restaurant attributes, and describe a novel methodology that can be scalably reused to generate NLG datasets for other domains. The experiments show that the models control im- portant aspects, including lexical choice of ad- jectives, output length, and sentiment, allow- ing the models to successfully hit multiple style targets without sacrificing semantics. 1 Introduction The increasing popularity of personal assistant di- alog systems and the success of end-to-end neu- ral models on problems such as machine transla- tion has lead to a surge of interest around data-to- text neural natural language generation (NNLG). State-of-the-art NNLG models commonly use a sequence-to-sequence framework for end-to-end neural language generation, taking a meaning rep- resentation (MR) as input, and generating a natu- ral language (NL) realization as output (Dusek and Jurc´ıcek, 2016; Lampouras and Vlachos, 2016; Mei et al., 2015; Wen et al., 2015b). Table 1 shows some examples of MR to human and system NL re- alizations from recently popular NNLG datasets. The real power of NNLG models over tradi- tional statistical generators is their ability to pro- duce natural language output from structured input in a completely data-driven way, without needing hand-crafted rules or templates. However, these models suffer from two critical bottlenecks: (1) a data bottleneck, i.e. the lack of large parallel training data of MR to NL, and (2) a control bot- tleneck, i.e. the inability to systematically control important aspects of the generated output to allow for more stylistic variation. Recent efforts to address the data bottleneck with large corpora for training neural generators have relied almost entirely on high-effort, costly crowdsourcing, asking humans to write references given an input MR. Table 1 shows two recent ef- forts: the E2E NLG …

Content truncated to 500 words