Neural Text Generation in Stories Using Entity Representations as Context

Proceedings of NAACL-HLT 2018, pages 2250–2260 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Neural Text Generation in Stories Using Entity Representations as Context Elizabeth Clark Yangfeng Ji Noah A. Smith Paul G. Allen School of Computer Science & Engineering University of Washington Seattle, WA, USA {eaclark7,yangfeng,nasmith}@cs.washington.edu Abstract We introduce an approach to neural text gen- eration that explicitly represents entities men- tioned in the text. Entity representations are vectors that are updated as the text proceeds; they are designed specifically for narrative text like fiction or news stories. Our experiments demonstrate that modeling entities offers a benefit in two automatic evaluations: mention generation (in which a model chooses which entity to mention next and which words to use in the mention) and selection between a correct next sentence and a distractor from later in the same story. We also conduct a human evalu- ation on automatically generated text in story contexts; this study supports our emphasis on entities and suggests directions for further re- search. 1 Introduction We consider the problem of automatically gen- erating narrative text, a challenging problem at the junction of computational creativity and lan- guage technologies (Gerv´as, 2009). We are moti- vated in particular by potential applications in per- sonalized education and assistive tools for human authors, though we believe narrative might also play a role in social conversational agents (Sor- doni et al., 2015). In this work, the term “narra- tive text” refers primarily to fiction but might also include news and other kinds of stories. A notable difference between longstanding work in natural language generation and recent “neural” models is in the treatment of entities and the words used to refer to them. Particularly in the generation of narrative text, character-centered generation has been shown important in character dialogue generation (Walker et al., 2011; Cavazza and Charles, 2005) and story planning (Cavazza et al., 2002). Neural models, on the other hand, treat mentions as just more words, relying on rep- resentation learning to relate the people in a story through the words alone. Context All of a sudden, [Emily]1 walked towards [the dragon]2. Current Sentence [Seth]3 yelled at [her]1 to get back but Figure 1: An example of entity-labeled story data. The brackets indicate which words are part of entity men- tions. Mentions marked with the same number refer to the same entity. The goal is to continue the story in a coherent way. The actual story reads, “Seth yelled at her to get back but she ignored him.” Entities are an important element of narrative text. Centering Theory places entities at the cen- ter of explaining what makes text coherent (Grosz et al., 1995). In this work, we incorporate entities into neural text generation models; each entity in a story is given its own vector representation, which is updated as the story unfolds. These represen- tations are learned specifically to predict words— both mentions of the entity itself and also the fol- lowing context. At a given …

Content truncated to 500 words