Learning distributed event representations with a multi-task approach

Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (*SEM), pages 11–21 New Orleans, June 5-6, 2018. c⃝2018 Association for Computational Linguistics Learning Distributed Event Representations with a Multi-Task Approach Xudong Hong†, Asad Sayeed*, Vera Demberg† †Dept. of Language Science and Technology, Saarland University {xhong,vera}@coli.uni-saarland.de *Dept. of Philosophy, Linguistics, and Theory of Science, University of Gothenburg asad.sayeed@gu.se Abstract Human world knowledge contains informa- tion about prototypical events and their partic- ipants and locations. In this paper, we train the first models using multi-task learning that can both predict missing event participants and also perform semantic role classification based on semantic plausibility. Our best-performing model is an improvement over the previous state-of-the-art on thematic fit modelling tasks. The event embeddings learned by the model can additionally be used effectively in an event similarity task, also outperforming the state- of-the-art. 1 Introduction Event representations consist, at minimum, of a predicate, the entities that participate in the event, and the thematic roles of those participants (Fill- more, 1968). The cook cut the cake with the knife expresses an event of cutting in which a cook is the “agent”, the cake is the “patient”, and the knife is the “instrument” of the action. Experiments have shown that event knowledge, in terms of the proto- typical participants of events and their structured compositions, plays a crucial role in human sen- tence processing, especially from the perspective of thematic fit: the extent to which humans per- ceive given event participants as “fitting” given predicate-role combinations (Ferretti et al., 2001; McRae et al., 2005; Bicknell et al., 2010). There- fore, computational models of language process- ing should also consist of event representations that reflect thematic fit. To evaluate this aspect em- pirically, a popular approach in previous work has been to compare model output to human judge- ments (Sayeed et al., 2016). The best-performing recent work has been the model of Tilk et al. (2016), who effectively simu- late thematic fit via selectional preferences: gener- ating a probability distribution over the full vocab- ulary of potential role-fillers. Given event context as input, including a predicate and a given set of semantic roles and their role-fillers as well as one target role, its training objective is to predict the correct role-filler for the target role. The objective of predicting upcoming role-fillers is cognitively plausible: there is ample evidence that humans an- ticipate upcoming input during sentence process- ing and learn from prediction error (Kuperberg and Jaeger, 2016; Friston, 2010) (even if other de- tails of the implementation like back-propagation may not have much to do with how errors are prop- agated in humans). An analysis of role filler predictions by Tilk et al.’s model shows that the model does not make sufficient use of the thematic role input. For in- stance, the representation of apple eats boy is sim- ilar to the representation of boy eats apple, even though the events are very dissimilar from one an- other. Interestingly, humans have been found to make similar …

Content truncated to 500 words