PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 4274–4295, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 4274 PLOTMACHINES: Outline-Conditioned Generation with Dynamic Plot State Tracking Hannah Rashkin1, Asli Celikyilmaz2, Yejin Choi1,3, Jianfeng Gao2 1 Paul G. Allen School of Computer Science & Engineering, University of Washington 2 Microsoft Research, Redmond, WA, USA 3 Allen Institute for Artificial Intelligence, Seattle, WA, USA {hrashkin,yejin}@cs.washington.edu, {aslicel,jfgao}@microsoft.com Abstract We propose the task of outline-conditioned story generation: given an outline as a set of phrases that describe key characters and events to appear in a story, the task is to generate a coherent narrative that is consistent with the provided outline. This task is challenging as the input only provides a rough sketch of the plot, and thus, models need to generate a story by interweaving the key points provided in the outline. This requires the model to keep track of the dynamic states of the latent plot, condi- tioning on the input outline while generating the full story. We present PLOTMACHINES, a neural narrative model that learns to trans- form an outline into a coherent story by track- ing the dynamic plot states. In addition, we enrich PLOTMACHINES with high-level dis- course structure so that the model can learn different writing styles corresponding to dif- ferent parts of the narrative. Comprehensive experiments over three fiction and non-fiction datasets demonstrate that large-scale language models, such as GPT-2 and GROVER, despite their impressive generation performance, are not sufficient in generating coherent narratives for the given outline, and dynamic plot state tracking is important for composing narratives with tighter, more consistent plots. 1 Introduction Composing a story requires a complex planning process. First, the writer starts with a rough sketch of what key characters and events the story will contain. Then, as they unfold the story, the writer must keep track of the elaborate plot that weaves together the characters and events in a coherent and consistent narrative. We study this complex storytelling process by formulating it as the task of outline-conditioned story generation, illustrated in Figure 1. Given an outline, a set of phrases describing key char- acters and events to appear in a story, the task is • big bird's birthday celebration • cookie monster eats • roller skating rink • big birthday cake It is Big Bird's birthday, and he goes to the roller skating rink with his friends. Back at Sesame Street, Maria and Susan take out the big birthday cake and leave it on a table. Cookie Monster sees the cake, but instead of eating it and spoiling the party, he eats a chair and other things all over Sesame Street. Big Bird and the other skaters return to Sesame Street and are shocked at what Cookie Monster ate, though the cake is safe. Gina and Count Von Count presents the cake to Big Bird. It has 548 candles even though Big Bird is 6 years old. At the end, when Gina announces the sponsors, Cookie …

Content truncated to 500 words