Skip N-grams and Ranking Functions for Predicting Script Events

Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 336–344, Avignon, France, April 23 - 27 2012. c⃝2012 Association for Computational Linguistics Skip N-grams and Ranking Functions for Predicting Script Events Bram Jans KU Leuven Leuven, Belgium bram.jans@gmail.com Steven Bethard University of Colorado Boulder Boulder, Colorado, USA steven.bethard@colorado.edu Ivan Vuli´c KU Leuven Leuven, Belgium ivan.vulic@cs.kuleuven.be Marie Francine Moens KU Leuven Leuven, Belgium sien.moens@cs.kuleuven.be Abstract In this paper, we extend current state-of-the- art research on unsupervised acquisition of scripts, that is, stereotypical and frequently observed sequences of events. We design, evaluate and compare different methods for constructing models for script event predic- tion: given a partial chain of events in a script, predict other events that are likely to belong to the script. Our work aims to answer key questions about how best to (1) identify representative event chains from a source text, (2) gather statistics from the event chains, and (3) choose ranking functions for predicting new script events. We make several contributions, introducing skip-grams for collecting event statistics, de- signing improved methods for ranking event predictions, defining a more reliable evalu- ation metric for measuring predictiveness, and providing a systematic analysis of the various event prediction models. 1 Introduction There has been recent interest in automatically ac- quiring world knowledge in the form of scripts (Schank and Abelson, 1977), that is, frequently recurring situations that have a stereotypical se- quence of events, such as a visit to a restaurant. All of the techniques so far proposed for this task share a common sub-task: given an event or partial chain of events, predict other events that belong to the same script (Chambers and Jurafsky, 2008; Chambers and Jurafsky, 2009; Chambers and Ju- rafsky, 2011; Manshadi et al., 2008; McIntyre and Lapata, 2009; McIntyre and Lapata, 2010; Regneri et al., 2010). Such a model can then serve as input to a system that identifies the order of the events within that script (Chambers and Jurafsky, 2008; Chambers and Jurafsky, 2009) or that generates a story using the selected events (McIntyre and Lapata, 2009; McIntyre and Lapata, 2010). In this article, we analyze and compare tech- niques for constructing models that, given a partial chain of events, predict other events that belong to the script. In particular, we consider the following questions: • How should representative chains of events be selected from the source text? • Given an event chain, how should statistics be gathered from it? • Given event n-gram statistics, which ranking function best predicts the events for a script? In the process of answering these questions, this article makes several contributions to the field of script and narrative event chain understanding: • We explore for the first time the use of skip- grams for collecting narrative event statistics, and show that this approach performs better than classic n-gram statistics. • We propose a new method for ranking events given a partial script, and show that it per- forms substantially better than ranking …

Content truncated to 500 words