Event Embeddings for Semantic Script Modeling

Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pages 75–83, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Event Embeddings for Semantic Script Modeling Ashutosh Modi Saarland University, Germany ashutosh@coli.uni-sb.de Abstract Semantic scripts is a conceptual represen- tation which defines how events are orga- nized into higher level activities. Practi- cally all the previous approaches to induc- ing script knowledge from text relied on count-based techniques (e.g., generative models) and have not attempted to compo- sitionally model events. In this work, we introduce a neural network model which relies on distributed compositional repre- sentations of events. The model captures statistical dependencies between events in a scenario, overcomes some of the short- comings of previous approaches (e.g., by more effectively dealing with data spar- sity) and outperforms count-based coun- terparts on the narrative cloze task. 1 Introduction It is generally believed that the lack of knowl- edge on how individual events are organized into higher-level scenarios is one of the major obstacles for natural language understanding. Texts often do not provide a detailed specification of underly- ing events as writers rely on the ability of humans to read between the lines or, more specifically, on their common sense knowledge of underlying sce- narios. For example, going to a restaurant involves entering the restaurant, getting seated, making an order and so on. Consequently, when describing a visit to a restaurant, a writer will not specify all the events, as they are obvious to the reader. This kind of knowledge is typically refereed to as se- mantic scripts (Schank and Abelson, 1977), and, in this work, will aim to capture some aspects of this knowledge within our probabilistic model. Early work on scripts focused on manual construction of knowledge bases and rule-based systems for inference using these knowledge bases (Schank and Abelson, 1977). More re- cent approaches relied on automatically learning script knowledge either from crowd-sourced or naturally-occurring texts (Chambers and Jurafsky, 2008; Regneri et al., 2010; Modi and Titov, 2014; Frermann et al., 2014; Jans et al., 2012; Pichotta and Mooney, 2014; Rudinger et al., 2015a). Most of these methods represent events as ver- bal predicates along with tuples of their immediate arguments (i.e. syntactic dependents of the pred- icate). These approaches model statistical depen- dencies between events (or, more formally, men- tions of events) in a document, often restricting their model to capturing dependencies only be- tween events sharing at least one entity (a common protagonist). We generally follow this tradition in our approach. Much of this previous work has focused on count-based techniques using, for example, either the generative framework (Frermann et al., 2014) or relying on information-theoretic measures such as pointwise mutual information (PMI) (Chambers and Jurafsky, 2008). Some of these techniques treat predicate-argument structures as an atomic whole (e.g., Pichotta and Mooney (2014)), in other words their probability estimates are based on co- occurrences of entire (predicate, arguments) tu- ples. Clearly such methods fail to adequately take into account compositional nature of …

Content truncated to 500 words