Inducing Neural Models of Script Knowledge

Proceedings of the Eighteenth Conference on Computational Language Learning, pages 49–57, Baltimore, Maryland USA, June 26-27 2014. c⃝2014 Association for Computational Linguistics Inducing Neural Models of Script Knowledge Ashutosh Modi MMCI, Saarland University, Germany amodi@mmci.uni-saarland.de Ivan Titov ILLC, University of Amsterdam, Netherlands titov@uva.nl Abstract Induction of common sense knowledge about prototypical sequence of events has recently received much attention (e.g., Chambers and Jurafsky (2008); Regneri et al. (2010)). Instead of inducing this knowledge in the form of graphs, as in much of the previous work, in our method, distributed representations of event real- izations are computed based on distributed representations of predicates and their ar- guments, and then these representations are used to predict prototypical event or- derings. The parameters of the composi- tional process for computing the event rep- resentations and the ranking component of the model are jointly estimated. We show that this approach results in a sub- stantial boost in performance on the event ordering task with respect to the previous approaches, both on natural and crowd- sourced texts. 1 Introduction It is generally believed that natural language un- derstanding systems would benefit from incorpo- rating common-sense knowledge about prototyp- ical sequences of events and their participants. Early work focused on structured representations of this knowledge (called scripts (Schank and Abelson, 1977)) and manual construction of script knowledge bases. However, these approaches do not scale to complex domains (Mueller, 1998; Gordon, 2001). More recently, automatic induc- tion of script knowledge from text have started to attract attention: these methods exploit ei- ther natural texts (Chambers and Jurafsky, 2008, 2009) or crowdsourced data (Regneri et al., 2010), and, consequently, do not require expensive ex- pert annotation. Given a text corpus, they ex- tract structured representations (i.e. graphs), for example chains (Chambers and Jurafsky, 2008) or more general directed acyclic graphs (Regneri et al., 2010). These graphs are scenario-specific, nodes in them correspond to events (and associ- ated with sets of potential event mentions) and arcs encode the temporal precedence relation. These graphs can then be used to inform NLP applica- tions (e.g., question answering) by providing in- formation whether one event is likely to precede or succeed another. Note that these graphs en- code common-sense knowledge about prototypi- cal ordering of events rather than temporal order of events as described in a given text. Though representing the script knowledge as graphs is attractive from the human interpretability perspective, it may not be optimal from the appli- cation point of view. More specifically, these rep- resentations (1) require a model designer to choose an appropriate granularity of event mentions (e.g., whether nodes in the graph should be associated with verbs, or also their arguments); (2) do not provide a mechanism for deciding which scenario applies in a given discourse context and (3) often do not associate confidence levels with informa- tion encoded in the graph (e.g., the precedence re- lation in Regneri et al. (2010)). Instead of constructing a graph and using it to provide information (e.g., prototypical event …

Content truncated to 500 words