Event Representation Learning Enhanced with External Commonsense Knowledge

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4894–4903, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4894 Event Representation Learning Enhanced with External Commonsense Knowledge Xiao Ding, Kuo Liao, Ting Liu∗, Zhongyang Li, Junwen Duan Research Center for Social Computing and Information Retrieval Harbin Institute of Technology, China {xding, kliao, tliu, zyli, jwduan}@ir.hit.edu.cn Abstract Prior work has proposed effective methods to learn event representations that can capture syntactic and semantic information over text corpus, demonstrating their effectiveness for downstream tasks such as script event predic- tion. On the other hand, events extracted from raw texts lacks of commonsense knowledge, such as the intents and emotions of the event participants, which are useful for distinguish- ing event pairs when there are only subtle dif- ferences in their surface realizations. To ad- dress this issue, this paper proposes to leverage external commonsense knowledge about the intent and sentiment of the event. Experiments on three event-related tasks, i.e., event simi- larity, script event prediction and stock mar- ket prediction, show that our model obtains much better event embeddings for the tasks, achieving 78% improvements on hard similar- ity task, yielding more precise inferences on subsequent events under given contexts, and better accuracies in predicting the volatilities of the stock market1. 1 Introduction Events are a kind of important objective informa- tion of the world. Structuralizing and represent- ing such information as machine-readable knowl- edge are crucial to artificial intelligence (Li et al., 2018b, 2019). The main idea is to learn distributed representations for structured events (i.e. event embeddings) from text, and use them as the basis to induce textual features for downstream appli- cations, such as script event prediction and stock market prediction. Parameterized additive models are among the most widely used for learning distributed event representations in prior work (Granroth-Wilding ∗Corresponding author 1The code and data are available on https://github.com/MagiaSN/CommonsenseERL EMNLP 2019. PersonX threw basketball PersonZ attacked embassy Intent PersonY threw bomb PersonZ attacked embassy PersonX broke record PersonY broke vase Sentiment PersonX broke record PersonY broke vase (a) (b) PersonY threw bomb PersonX threw basketball PersonZ achieved goal PersonZ achieved goal Figure 1: Intent and sentiment enhanced event embed- dings can distinguish distinct events even with high lex- ical overlap, and find similar events even with low lex- ical overlap. and Clark, 2016; Modi, 2016), which passes the concatenation or addition of event arguments’ word embeddings to a parameterized function. The function maps the summed vectors into an event embedding space. Furthermore, Ding et al. (2015) and Weber et al. (2018) propose using neu- ral tensor networks to perform semantic composi- tion of event arguments, which can better capture the interactions between event arguments. This line of work only captures shallow event semantics, which is not capable of distinguishing events with subtle differences. On the one hand, the obtained event embeddings cannot capture the relationship between events that are syntactically or semantically …

Content truncated to 500 words