COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completion

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5086–5099 August 1–6, 2021. ©2021 Association for Computational Linguistics 5086 COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completion Debjit Paul Research Training Group AIPHES Institute for Computational Linguistics Heidelberg University paul@cl.uni-heidelberg.de Anette Frank Research Training Group AIPHES Institute for Computational Linguistics Heidelberg University frank@cl.uni-heidelberg.de Abstract Despite recent successes of large pre-trained language models in solving reasoning tasks, their inference capabilities remain opaque. We posit that such models can be made more inter- pretable by explicitly generating interim infer- ence rules, and using them to guide the gener- ation of task-specific textual outputs. In this paper we present COINS, a recursive inference framework that i) iteratively reads context sen- tences, ii) dynamically generates contextual- ized inference rules, encodes them, and iii) uses them to guide task-specific output gener- ation. We apply COINS to a Narrative Story Completion task that asks a model to complete a story with missing sentences, to produce a coherent story with plausible logical connec- tions, causal relationships, and temporal de- pendencies. By modularizing inference and sentence generation steps in a recurrent model, we aim to make reasoning steps and their ef- fects on next sentence generation transparent. Our automatic and manual evaluations show that the model generates better story sentences than SOTA baselines, especially in terms of coherence. We further demonstrate improved performance over strong pre-trained LMs in generating commonsense inference rules. The recursive nature of COINS holds the potential for controlled generation of longer sequences. 1 Introduction Narrative story understanding, and similarly story generation, requires the ability to construe mean- ing that is not explicitly stated through common- sense reasoning over events in the story (Rashkin et al., 2018a). Previous work in modeling narrative stories has focused on learning scripts1 (Schank and Abelson, 1977; Mooney and DeJong, 1985) and learning narrative schemas using corpus statis- 1Scripts are structured knowledge about stereotypical event sequences together with their participants. Beginning: S1: Janie was excited to see her sister's play in theatre. S2: Janie got a call from her boss about an emergency work. • SomeoneA wasn’t able to go SomewhereB (to see the play) End: S5: Janie watched a video of the play later. • SomeoneA wants to go to SomewhereB (to theatre) • SomeoneA possess(es) a phone. • SomeoneB wants SomeoneA to work. Implicit Inference Rules Effect Effect and Cause Cause Context : S3: Janie’s boss gave her new work. S4: Janie couldn’t attend her sisters’ play Missing Sentence: Figure 1: An example of the Narrative Story Comple- tion Task. Top and bottom boxes show the context (top) and missing sentences (bottom). The chain of implicit inference rules explains the connection between begin- ning and end, and allows to infer the missing sentences. tics (Chambers and Jurafsky, 2009; Balasubrama- nian et al., 2013; Nguyen et al., 2015). Recently, large pretrained language models (LMs) such as GPT-2 have shown remarkable performance on var- …

Content truncated to 500 words