Event Representation with Sequential, Semi-Supervised Discrete Variables

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4701–4716 June 6–11, 2021. ©2021 Association for Computational Linguistics 4701 Event Representation with Sequential, Semi-Supervised Discrete Variables Mehdi Rezaee Department of Computer Science University of Maryland Baltimore County Baltimore, MD 21250 USA rezaee1@umbc.edu Francis Ferraro Department of Computer Science University of Maryland Baltimore County Baltimore, MD 21250 USA ferraro@umbc.edu Abstract Within the context of event modeling and understanding, we propose a new method for neural sequence modeling that takes partially-observed sequences of discrete, ex- ternal knowledge into account. We construct a sequential neural variational autoencoder, which uses Gumbel-Softmax reparametriza- tion within a carefully defined encoder, to allow for successful backpropagation during training. The core idea is to allow semi- supervised external discrete knowledge to guide, but not restrict, the variational latent pa- rameters during training. Our experiments in- dicate that our approach not only outperforms multiple baselines and the state-of-the-art in narrative script induction, but also converges more quickly. 1 Introduction Event scripts are a classic way of summarizing events, participants, and other relevant information as a way of analyzing complex situations (Schank and Abelson, 1977). To learn these scripts we must be able to group similar-events together, learn common patterns/sequences of events, and learn to represent an event’s arguments (Minsky, 1974). While continuous embeddings can be learned for events and their arguments (Ferraro et al., 2017; Weber et al., 2018a), the direct inclusion of more structured, discrete knowledge is helpful in learn- ing event representations (Ferraro and Van Durme, 2016). Obtaining fully accurate structured knowl- edge can be difficult, so when the external knowl- edge is neither sufficiently reliable nor present, a natural question arises: how can our models use the knowledge that is present? Generative probabilistic models provide a frame- work for doing so: external knowledge is a random variable, which can be observed or latent, and the data/observations are generated (explained) from it. Knowledge that is discrete, sequential, or both— Crush occurred at station <latexit sha1_base64="7Hk7rU2krBpGatMqW9ME3zVmSls=">AB6nicdVDLSgNBEOyNrxhfUY9eBoPgaZlNzK65Bb14jGhiIFnC7GQ2GTL7YGZWCG f4MWDIl79Im/+jZOHoKIFDUVN91dQSq40h/WLmV1bX1jfxmYWt7Z3evuH/QUkmKWvSRCSyHRDFBI9ZU3MtWDuVjESBYHfB6HLm390zqXgS3+pxyvyIDGIeckq0kW7CXrlXLGHb8bDr1BC2K27FrWFDPK9cdTFybDxHCZo9Irv3X5Cs4jFmgqiVMfBqfYnRGpOBZsWupliKaEjMmAdQ2MSMeVP5qdO0YlR+ihMpKlY o7n6fWJCIqXGUWA6I6KH6rc3E/yOpkOz/0Jj9NMs5guFoWZQDpBs79Rn0tGtRgbQqjk5lZEh0QSqk06BRPC16fof9Iq207VxtdnpfrFMo48HMExnIDHtThChrQBAoDeIAneLaE9Wi9WK+L1py1nDmEH7DePgFj543h</latexit>f2 <latexit sha1_base64="mGIOp0nJe8b5O7mrOeWy9IWt7s=">AB6nicdVDJSgNBEK2JW4xb1KOXxiB4GnomCSa3oBePEc0CyRB6Oj1Jk56F7h4hDPk ELx4U8eoXefNv7CyCij4oeLxXRVU9PxFcaYw/rNza+sbmVn67sLO7t39QPDxqziVlLVoLGLZ9YligkespbkWrJtIRkJfsI4/uZr7nXsmFY+jOz1NmBeSUcQDTok20m0wKA+KJWzX67hSqSJsV7HrujVDcNmt1R3k2HiBEqzQHBTf+8OYpiGLNBVEqZ6DE+1lRGpOBZsV+qliCaETMmI9QyMSMuVli1Nn6MwoQxTE0lSk 0UL9PpGRUKlp6JvOkOix+u3Nxb+8XqDmpfxKEk1i+hyUZAKpGM0/xsNuWRUi6khEpubkV0TCSh2qRTMCF8fYr+J23Xdqo2vqmUGperOPJwAqdwDg5cQAOuoQktoDCB3iCZ0tYj9aL9bpszVmrmWP4AevtE16Djd0=</latexit>f3 Train collided with train Killed passengers and injured <latexit sha1_base64="yIL6yp3fIwMcYUAM5v+oQdlkmk=">AB+3icdVDLSgMxFM3UV62vWpdugkVwNWTGFtd0Y3gpoJ9QDuUTJq2oZkH yR1pGforblwo4tYfcefmD4EFT1w4XDOvcm9x4+l0EDIh5VZW9/Y3Mpu53Z29/YP8oeFpo4SxXiDRTJSbZ9qLkXIGyBA8nasOA18yVv+Grut+650iIK72Aacy+gw1AMBKNgpF6+0OkCn4Bm6Y2Q5pHhzOvli8SuVkmpVMbELhPXdSuGkHO3UnWwY5MFimiFei/3u1HLAl4CExSrTsOicFLqQLBJ/l uonmMWVjOuQdQ0MacO2li91n+NQofTyIlKkQ8EL9PpHSQOtp4JvOgMJI/bm4l9eJ4FBxUtFGCfAQ7b8aJBIDBGeB4H7QnEGcmoIZUqYXTEbUZmLhyJoSvS/H/pOnaTtkmt6Vi7XIVRxYdoxN0hx0gWroGtVRAzE0Q/oCT1bM+vRerFel60ZazVzhH7AevsEDtaVHQ=</latexit>[Killing] Figure 1: An overview of event modeling, where the ob- served events (black text) are generated via a sequence of semi-observed random variables. In this case, the random variables are directed to take on the meaning of semantic frames that can be helpful to explain the events. Unobserved frames are in orange and observed frames are in blue. Some connections are more impor- tant than others, indicated by the weighted arrows. such as for script learning—complicates the devel- opment of neural generative models. In this paper, we provide a successful approach for incorporating partially-observed, discrete, se- quential external knowledge in a neural genera- tive model. We specifically examine event se- quence modeling augmented by semantic frames. Frames (Minsky, 1974, i.a.) are a semantic repre- sentation designed to capture the common and gen- eral knowledge we have about events, situations, and things. They have …

Content truncated to 500 words