Conditional Generation of Temporally-ordered Event Sequences

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 7142–7157 August 1–6, 2021. ©2021 Association for Computational Linguistics 7142 Conditional Generation of Temporally-ordered Event Sequences Shih-Ting Lin♠ Nathanael Chambers♦ Greg Durrett♠ ♠ The University of Texas at Austin ♦ United States Naval Academy j0717lin@cs.utexas.edu, nchamber@usna.edu, gdurrett@cs.utexas.edu Abstract Models of narrative schema knowledge have proven useful for a range of event-related tasks, but they typically do not capture the tempo- ral relationships between events. We pro- pose a single model that addresses both tem- poral ordering, sorting given events into the order they occurred, and event infilling, pre- dicting new events which fit into an existing temporally-ordered sequence. We use a BART- based conditional generation model that can capture both temporality and common event co-occurrence, meaning it can be flexibly ap- plied to different tasks in this space. Our model is trained as a denoising autoencoder: we take temporally-ordered event sequences, shuffle them, delete some events, and then at- tempt to recover the original event sequence. This task teaches the model to make infer- ences given incomplete knowledge about the events in an underlying scenario. On the tem- poral ordering task, we show that our model is able to unscramble event sequences from existing datasets without access to explicitly labeled temporal training data, outperform- ing both a BERT-based pairwise model and a BERT-based pointer network. On event infill- ing, human evaluation shows that our model is able to generate events that fit better tempo- rally into the input events when compared to GPT-2 story completion models. 1 Introduction This paper proposes a single model of events to support inferences in two seemingly different tasks: (1) temporal event ordering and (2) event infilling, or inferring unseen or unmentioned events occur- ring as part of a larger scenario. Figure 1 shows an example illustrating these two goals. Unlike prior approaches, we aim to address both with the same model architecture, rather than having to annotate data and build ad-hoc models for each task sepa- rately; our goal is to work towards models that cap- Temporal BART I opened a present She gave me the present She bought the present Complete ordered event sequence She bought the present I opened a present Scrambled input events Figure 1: Diagram of our modeling setup: Temporal- BART captures both temporal ordering and event cooc- currence to make various event-related inferences. ture temporal event knowledge broadly and support a wide range of inferences. We thus need a suitably general modeling framework to capture temporal knowledge about events, which in our case will be a BART-based (Lewis et al., 2020) model we call TemporalBART. Note that classic temporal relation extraction models, which model temporal ordering in context for a particular document, may chiefly learn how to use local discourse cues rather than generalizable event knowledge (Chambers et al., 2014; Ning et al., 2018b). The goals in this work relate to past work on learning narrative …

Content truncated to 500 words