Strategies for Structuring Story Generation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2650–2660 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2650 Strategies for Structuring Story Generation Angela Fan FAIR, Paris LORIA, Nancy angelafan@fb.com Mike Lewis FAIR, Seattle mikelewis@fb.com Yann Dauphin Google AI∗ ynd@google.com Abstract Writers often rely on plans or sketches to write long stories, but most current language models generate word by word from left to right. We explore coarse-to-fine models for creating nar- rative texts of several hundred words, and in- troduce new models which decompose stories by abstracting over actions and entities. The model first generates the predicate-argument structure of the text, where different mentions of the same entity are marked with placeholder tokens. It then generates a surface realiza- tion of the predicate-argument structure, and finally replaces the entity placeholders with context-sensitive names and references. Hu- man judges prefer the stories from our models to a wide range of previous approaches to hi- erarchical text generation. Extensive analysis shows that our methods can help improve the diversity and coherence of events and entities in generated stories. 1 Introduction Stories exhibit structure at multiple levels. While existing language models can generate stories with good local coherence, they struggle to coalesce in- dividual phrases into coherent plots or even main- tain character consistency throughout a story. One reason for this failure is that classical language models generate the whole story at the word level, which makes it difficult to capture the high-level interactions between the plot points. To address this, we investigate novel decompo- sitions of the story generation process that break down the problem into a series of easier coarse-to- fine generation problems. These decompositions can offer three advantages: • They allow more abstract representations to be generated first, where challenging long- range dependencies may be more apparent. *Work done while at Facebook AI Research Figure 1: Proposed Model. Conditioned upon the prompt, we generate sequences of predicates and ar- guments. Then, a story is generated with placeholder entities such as ent0. Finally we replace the placehold- ers with specific references. • They allow specialized modelling techniques for the different stages, which exploit the structure of the specific sub-problem. • They are applicable to any textual dataset and require no manual labelling. Several hierarchical models for story generation have recently been proposed (Xu et al., 2018; Yao et al., 2019), but it is not well understood which properties characterize a good decomposition. We therefore implement and evaluate several repre- sentative approaches based on keyword extraction, sentence compression, and summarization. We build on this understanding to devise the proposed decomposition (Figure 1). Inspired by the classic model of Reiter and Dale (2000), our approach breaks down the generation process in three steps: modelling the action sequence, the story narrative, and lastly entities such as story characters. To model action sequences, we first generate the predicate-argument structure of the 2651 story by generating a sequence of verbs and ar- guments. This representation …

Content truncated to 500 words