Narrative Embedding: Re-Contextualization Through Attention

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1393–1405 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1393 Narrative Embedding: Re-Contextualization through Attention Sean Wilner Vody LLC sean@vody.com Daniel Woolridge Vody LLC dan@vody.com Madeleine Glick Vody LLC madeleine@vody.com Abstract Narrative analysis is becoming increasingly important for a number of linguistic tasks in- cluding summarization, knowledge extraction, and question answering. We present a novel approach for narrative event representation us- ing attention to re-contextualize events across the whole story. Comparing to previous analy- sis we find an unexpected attachment of event semantics to predicate tokens within a popu- lar transformer model. We test the utility of our approach on narrative completion predic- tion, achieving state of the art performance on Multiple Choice Narrative Cloze and scoring competitively on the Story Cloze Task. 1 Introduction Common sequences of events describing real-world interactions are a fount of useful information for how we interact with our world. For example, the prototypical “restaurant schema" of Schank and Abelson (2013) might contain the sequence of events in Story A of Figure 1. Knowledge of this plausible sequence of events allows us to both pre- dict likely subsequent events in a novel ‘restaurant experience’ as well as infer unstated events when observing a recounting of the same. Since written stories often provide the minimum required for hu- man comprehension, the stated narrative is liable to be rife with omission, leaving the reader to infer the missing events. For this reason, understanding these chains of events is central to understanding the narrative as a whole. Each story is made up of constituent events, and by developing representations of events we can in turn produce a representation of the story at large. Event representation has been a topic of substantial interest to the field (Chambers and Jurafsky, 2008, 2009; Granroth-Wilding and Clark, 2016; Wang et al., 2017; Mostafazadeh et al., 2016b) and as the wealth of unstructured narrative data continues to grow, to industry as well. Indeed, industry has Figure 1: A selection of events from two story lines. Each narrative contains the event "John orders fish", however the meaning of the event within the context of each story is substantially different. seen narrative representations applied both directly in the case of film and book analysis (Scriptbook, 2021) as well as indirectly as a means of communi- cation in automated data explanation applications (Narrative Science, 2021). Our goal is to provide an unsupervised approach to narrative representation by utilizing and com- bining advances in representational techniques at both the semantic and structural level. Our ap- proach is based on ‘re-contextualization’, a process of embedding representations through transformer blocks at two levels of granularity. First, similarly to standard transformer models, we contextualize events within a single sentence to provide local context. Second, we extract events from each sen- tence to contextualize them across an entire story or story arc. The motivation for this approach is to allow our representation to differentiate between events which, though …

Content truncated to 500 words