Event participant modelling with neural networks

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 171–182, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Event participant modelling with neural networks Ottokar Tilk Institute of Cybernetics Tallinn University of Technology 12618 Tallinn, Estonia ottokar.tilk@phon.ioc.ee Vera Demberg and Asad Sayeed and Dietrich Klakow and Stefan Thater Saarland University 66123 Saarbr¨ucken, Germany {vera,asayeed,stth} @coli.uni-sb.de; dietrich.klakow@lsv.uni-sb.de Abstract A common problem in cognitive modelling is lack of access to accurate broad-coverage models of event-level surprisal. As shown in, e.g., Bicknell et al. (2010), event-level knowl- edge does affect human expectations for ver- bal arguments. For example, the model should be able to predict that mechanics are likely to check tires, while journalists are more likely to check typos. Similarly, we would like to predict what locations are likely for play- ing football or playing flute in order to esti- mate the surprisal of actually-encountered lo- cations. Furthermore, such a model can be used to provide a probability distribution over fillers for a thematic role which is not men- tioned in the text at all. To this end, we train two neural network mod- els (an incremental one and a non-incremental one) on large amounts of automatically role- labelled text. Our models are probabilistic and can handle several roles at once, which also enables them to learn interactions between dif- ferent role fillers. Evaluation shows a drastic improvement over current state-of-the-art sys- tems on modelling human thematic fit judge- ments, and we demonstrate via a sentence sim- ilarity task that the system learns highly useful embeddings. 1 Introduction Our goals in this paper are to learn a representa- tion of events and their thematic roles based on large quantities of automatically role-labelled text and to be able to calculate probability distributions over the possible role fillers of specific missing roles. In this sense, the task is closely related to work on selec- tional preference acquisition (Van de Cruys, 2014). We focus here on the roles agent, patient, loca- tion, time, manner and the predicate itself. The model we develop is trained to represent the event- relevant context and hence systematically captures long-range dependencies. This has been previously shown to be beneficial also for more general lan- guage modelling tasks (e.g., Chelba and Jelinek, 1998; Tan et al., 2012). This type of modelling is potentially relevant to a wide range of tasks, for instance for performing the- matic fit judgment tasks, detecting anomalous events (Dasigi and Hovy, 2014), or predicting event struc- ture that is not explicitly present in the text. The latter could be useful for inferring missing informa- tion in entailment tasks or improving identification of thematic roles outside the sentence containing the predicate. Potential applications also include predi- cate prediction based on arguments and roles, which has been noted to be relevant for simultaneous ma- chine translation for a verb-final to a verb-medial source language (Grissom II et al., 2014). Within cognitive modelling, our model could help to more accurately estimate semantic surprisal for …

Content truncated to 500 words