Sentence-Level Content Planning and Style Specification for Neural Text Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 591–602, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 591 Sentence-Level Content Planning and Style Specification for Neural Text Generation Xinyu Hua and Lu Wang Khoury College of Computer Sciences Northeastern University Boston, MA 02115 hua.x@husky.neu.edu luwang@ccs.neu.edu Abstract Building effective text generation systems re- quires three critical components: content se- lection, text planning, and surface realiza- tion, and traditionally they are tackled as sep- arate problems. Recent all-in-one style neu- ral generation models have made impressive progress, yet they often produce outputs that are incoherent and unfaithful to the input. To address these issues, we present an end-to- end trained two-step generation model, where a sentence-level content planner first decides on the keyphrases to cover as well as a de- sired language style, followed by a surface re- alization decoder that generates relevant and coherent text. For experiments, we consider three tasks from domains with diverse topics and varying language styles: persuasive ar- gument construction from Reddit, paragraph generation for normal and simple versions of Wikipedia, and abstract generation for scien- tific articles. Automatic evaluation shows that our system can significantly outperform com- petitive comparisons. Human judges further rate our system generated text as more fluent and correct, compared to the generations by its variants that do not consider language style. 1 Introduction Automatic text generation is a long-standing chal- lenging task, as it needs to solve at least three major problems: (1) content selection (“what to say”), identifying pertinent information to present, (2) text planning (“when to say what”), arrang- ing content into ordered sentences, and (3) surface realization (“how to say it”), deciding words and syntactic structures that deliver a coherent output based on given discourse goals (McKeown, 1985). Traditional text generation systems often handle each component separately, thus requiring exten- sive effort on data acquisition and system engi- neering (Reiter and Dale, 2000). Recent progress Topic: US should cut off foreign aid completely. /r/ChangeMyView Counter-argument: It can be a useful political bargaining chip. A few years ago, the US cut financial aid to Uganda due to its plans to make homosexuality a crime punishable by death. Please consider changing your mind! Topic: Artificial Intelligence English Wikipedia: … Computer science defines AI research as … any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals. ... Simple Wikipedia: Artificial Intelligence is the ability of a computer program or a machine to think and learn. ... Figure 1: [Upper] Sample counter-argument from Red- dit. Argumentative stylistic language for persuasion is in italics. [Bottom] Excerpts from Wikipedia, where sophisticated concepts and language of higher com- plexity used in the standard version are not present in the corresponding simplified version. Both: key con- cepts are in bold. has been made by developing end-to-end trained neural models (Rush et al., 2015; Yu et al., 2018; Fan et al., …

Content truncated to 500 words