Narrative Text Generation with a Latent Discrete Plan

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3637–3650 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3637 Narrative Text Generation with a Latent Discrete Plan Harsh Jhamtani 1 Taylor Berg-Kirkpatrick 2 1 School of Computer Science, Carnegie Mellon University 2 Computer Science and Engineering. University of California San Diego jharsh@cs.cmu.edu, tberg@ucsd.eng.edu Abstract Past work on story generation has demon- strated the usefulness of conditioning on a gen- eration plan to generate coherent stories. How- ever, these approaches have used heuristics or off-the-shelf models to first tag training sto- ries with the desired type of plan, and then train generation models in a supervised fash- ion. In this paper, we propose a deep latent variable model that first samples a sequence of anchor words, one per sentence in the story, as part of its generative process. During train- ing, our model treats the sequence of anchor words as a latent variable and attempts to in- duce anchoring sequences that help guide gen- eration in an unsupervised fashion. We con- duct experiments with several types of sen- tence decoder distributions – left-to-right and non-monotonic, with different degrees of re- striction. Further, since we use amortized vari- ational inference to train our model, we in- troduce two corresponding types of inference network for predicting the posterior on anchor words. We conduct human evaluations which demonstrate that the stories produced by our model are rated better in comparison with base- lines which do not consider story plans, and are similar or better in quality relative to base- lines which use external supervision for plans. Additionally, the proposed model gets favor- able scores when evaluated on perplexity, di- versity, and control of story via discrete plan. 1 Introduction Maintaining long-term narrative flow and consis- tency are important concerns when aiming to gener- ate a plausible story (Porteous and Cavazza, 2009; Hou et al., 2019). Prior work on narrative text gen- eration has focused on generating consistent stories via story outlines using keywords or key phrases (Xu et al., 2018; Yao et al., 2019), event-based representations (Riedl and Young, 2010; Martin et al., 2018; Fan et al., 2019), plot graphs (Li et al., Figure 1: Our aim is to generate a story given a title. We propose models which first generate a high level story plan realized via a sequence of anchor words. 2013) or a sentence representing theme (Chen et al., 2019). Yao et al. (2019) note that compared to specific event based representations, using keywords to form the outline is more generalizable and widely applicable. In this work, we consider a sequence of anchor words as a means to model story out- lines. For example, in Figure 1, given a story title ‘Winning the Race’, our model first predicts a se- quence of anchor words which represents a high level story plan. Thereafter, a decoder conditions on the title and generated sequence of anchor words to generate the final story. We assume an alignment between the anchor words and …

Content truncated to 500 words