Generating Narrative Text in a Switching Dynamical System

Proceedings of the 24th Conference on Computational Natural Language Learning, pages 520–530 Online, November 19-20, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 520 Generating Narrative Text in a Switching Dynamical System Noah Weber 3,1, Leena Shekhar 1∗, Heeyoung Kwon1 Niranjan Balasubramanian1, Nathanael Chambers 2 1Stony Brook University {lshekhar,heekwon,niranjan}@cs.stonybrook.edu 2United States Naval Academy nchamber@usna.edu 3Johns Hopkins University nweber6@jhu.edu Abstract Early work on narrative modeling used explicit plans and goals to generate stories, but the lan- guage generation itself was restricted and in- flexible. Modern methods use language mod- els for more robust generation, but often lack an explicit representation of the scaffolding and dynamics that guide a coherent narrative. This paper introduces a new model that inte- grates explicit narrative structure with neural language models, formalizing narrative model- ing as a Switching Linear Dynamical System (SLDS). A SLDS is a dynamical system in which the latent dynamics of the system (i.e. how the state vector transforms over time) is controlled by top-level discrete switching vari- ables. The switching variables represent nar- rative structure (e.g., sentiment or discourse states), while the latent state vector encodes information on the current state of the narra- tive. This probabilistic formulation allows us to control generation, and can be learned in a semi-supervised fashion using both labeled and unlabeled data. Additionally, we derive a Gibbs sampler for our model that can “fill in” arbitrary parts of the narrative, guided by the switching variables. Our filled-in (English lan- guage) narratives outperform several baselines on both automatic and human evaluations. 1 Introduction A narrative is a textualized sequence of events that serves as a coherent outline for an actual story (Prince, 2003). Effective narratives are typ- ically built on top of higher level narrative scaf- folds1 which specify at an abstract level how the story should evolve along different dimensions. Ex- ample scaffolds include descriptions of the emo- tional trajectory of a story (Vonnegut, 1981; Rea- gan et al., 2016), the goals of characters throughout ∗Author now at Microsoft 1We use the term scaffold as an umbrella term to cover many types of plans and structures that underlie stories. Tom didn’t know why his internet speed was so slow. Tom wasn’t sure what to do with his computer. He thought he would fix it himself. Tom was surprisingly good. Tom was happy to be surfing the internet again Table 1: A sample filled in narrative generated by our SLDS model given the first and last sentences as input (grayed out), the middle 3 sentences are imputed by our model (bold). the story (Meehan, 1977; Turner, 1993), or the ab- stract types of events that may occur (Martin et al., 2018). The parts of a scaffold are generic, and like Propp’s originally proposed narrative functions (Propp, 1928), can be reused across stories. To be fully reusable, one needs to go beyond just identi- fying what the elements of the narrative scaffold are, and also indicate how each scaffold element changes the properties of the current story state. We refer …

Content truncated to 500 words