Modeling Event Background for If-Then Commonsense Reasoning Using Context-aware Variational Autoencoder

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2682–2691, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2682 Modeling Event Background for If-Then Commonsense Reasoning Using Context-aware Variational Autoencoder Li Du, Xiao Ding, Ting Liu∗ and Zhongyang Li Research Center for Social Computing and Information Retrieval Harbin Institute of Technology, China {ldu, xding, tliu, zyli}@ir.hit.edu.cn Abstract Understanding event and event-centered com- monsense reasoning are crucial for natural lan- guage processing (NLP). Given an observed event, it is trivial for human to infer its in- tents and effects, while this type of If-Then reasoning still remains challenging for NLP systems. To facilitate this, a If-Then common- sense reasoning dataset Atomic is proposed, together with an RNN-based Seq2Seq model to conduct such reasoning. However, two fun- damental problems still need to be addressed: first, the intents of an event may be multiple, while the generations of RNN-based Seq2Seq models are always semantically close; sec- ond, external knowledge of the event back- ground may be necessary for understanding events and conducting the If-Then reasoning. To address these issues, we propose a novel context-aware variational autoencoder effec- tively learning event background information to guide the If-Then reasoning. Experimental results show that our approach improves the accuracy and diversity of inferences compared with state-of-the-art baseline methods. 1 Introduction Recently, event-centered commonsense knowl- edge has attracted much attention (Chambers and Jurafsky, 2008; Segers et al., 2016; Wang et al., 2017; Li et al., 2018), because of understanding events is an important component of NLP. Given a daily-life event, human can easily understand it and reason about its causes, effects, and so on. However, it still remains a challenging task for NLP systems. This is partly due to most of them are trained for task-specific datasets or objectives, which results in models that are adapt at finding task-specific underlying correlation patterns but have limited capability in simple and explainable commonsense reasoning (Sap et al., 2018). ∗Corresponding author Figure 1: A illustration of two challenging problems in If- Then reasoning. (a) Given an observed event, the feelings about this event could be multiple. (b) Background knowl- edge is need for generating reasonable inferences, which is absent in the dataset (marked by dashed lines). To facilitate this, Rashkin et al. (2018) build the Event2Mind dataset and Sap et al. (2018) present the Atomic dataset, mainly focus on nine If-Then reasoning types to describe causes, effects, intents and participant characteristic about events. To- gether with these datasets, a simple RNN-based encoder-decoder framework is proposed to con- duct the If-Then reasoning. However, there still remains two challenging problems. First, as illustrated in Figure 1, given an event “PersonX finds a new job”, the plausi- ble feeling of PersonX about that event could be multiple (such as “needy/stressed out” and “re- lieved/joyful”). Previous work showed that for the one-to-many problem, conventional RNN-based encoder-decoder models tend to generate generic responses, rather than meaningful and specific …

Content truncated to 500 words