CosMo: Conditional Seq2Seq-based Mixture Model for Zero-Shot Commonsense Question Answering

Proceedings of the 28th International Conference on Computational Linguistics, pages 5347–5359 Barcelona, Spain (Online), December 8-13, 2020 5347 COSMO: Conditional SEQ2SEQ-based Mixture Model for Zero-Shot Commonsense Question Answering Farhad Moghimifar1 and Lizhen Qu2 and Yue Zhuo3 Mahsa Baktashmotlagh1 and Gholamreza Haffari2 1The School of ITEE, The University of Queensland, Australia 2Faculty of Information Technology, Monash University, Australia 3School of CSE, The University of New South Wales, Australia {f.moghimifar,m.baktashmotlagh}@uq.edu.au firstname.lastname@monash.edu, terry.zhuo@unsw.edu.au Abstract Commonsense reasoning refers to the ability of evaluating a social situation and acting accordingly. Identification of the implicit causes and effects of a social context is the driving capability which can enable machines to perform commonsense reasoning. The dynamic world of social interactions requires context-dependent on-demand systems to infer such underlying information. However, current approaches in this realm lack the ability to perform commonsense reasoning upon facing an unseen situation, mostly due to incapability of identifying a diverse range of implicit social relations. Hence they fail to estimate the correct reasoning path. In this paper, we present Conditional SEQ2SEQ-based Mixture model (COSMO), which provides us with the capabilities of dynamic and diverse content generation. We use COSMO to generate context-dependent clauses, which form a dynamic Knowledge Graph (KG) on-the-fly for commonsense reasoning. To show the adaptability of our model to context-dependant knowledge generation, we address the task of zero-shot commonsense question answering. The empirical results indicate an improvement of up to +5.2% over the state-of-the-art models. 1 Introduction People understand narratives of everyday life by capitalising on their commonsense knowledge. They can easily reason about unobserved causes and effects in relation to the events described in narratives, as well as plausible characteristics and mental states of the involved persons. Although this kind of reasoning seems trivial for humans, it is still out of reach for current natural language understanding (NLU) systems. Recently, there have been fast-growing interests in building AI systems with such human-like reasoning capabilities based on inferential commonsense knowledge (Storks et al., 2019; Bosselut and Choi, 2019). Such systems are often evaluated by answering questions based on narratives. As illustrated in Figure 1, given a narrative “Austin often spends her weekend at the lake fishing with friends” and a question regarding the intention of Austin, an AI system is supposed to associate this event to relevant events in an inferential knowledge base or a web-scale corpus, find plausible reasons of those events, and conclude that “wanted to relax” is the most probable answer. Knowledge-based approaches to such commonsense question-answering (QA) require an inferential knowledge base. ATOMIC (Sap et al., 2019a) is the largest commonsense knowledge base of this kind, which contains 300,000 short textual description of events and 877,000 typed if-then relations between events, categorised into 9 dimensions. For instance, IF the event “X puts trust in Y” occurs and the target relation is “xWant”, THEN “X wants to develop a relationship”. Prior work utilizes knowledge in ATOMIC by formulating the learning problem as event prediction in if-then relations (Bosselut et al., 2019). In particular, they encode …

Content truncated to 500 words