Modularized Transfer Learning with Multiple Knowledge Graphs for Zero-shot Commonsense Reasoning

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2244 - 2257 July 10-15, 2022 ©2022 Association for Computational Linguistics Modularized Transfer Learning with Multiple Knowledge Graphs for Zero-shot Commonsense Reasoning Yu Jin Kim1, Beong-woo Kwak1, Youngwook Kim1, Reinald Kim Amplayo2∗ Seung-won Hwang3 and Jinyoung Yeo1† 1Yonsei University, Seoul, Korea 2Google Research, London, UK 3Seoul National University, Seoul, Korea {yujin000731, beongwoo.kwak, youngwook, jinyeo}@yonsei.ac.kr reinald@google.com, seungwonh@snu.ac.kr Abstract Commonsense reasoning systems should be able to generalize to diverse reasoning cases. However, most state-of-the-art approaches de- pend on expensive data annotations and over- fit to a specific benchmark without learning how to perform general semantic reasoning. To overcome these drawbacks, zero-shot QA systems have shown promise as a robust learn- ing scheme by transforming a commonsense knowledge graph (KG) into synthetic QA- form samples for model training. Considering the increasing type of different commonsense KGs, this paper aims to extend the zero-shot transfer learning scenario into multiple-source settings, where different KGs can be utilized synergetically. Towards this goal, we propose to mitigate the loss of knowledge from the interference among the different knowledge sources, by developing a modular variant of the knowledge aggregation as a new zero-shot commonsense reasoning framework. Results on five commonsense reasoning benchmarks demonstrate the efficacy of our framework, im- proving the performance with multiple KGs. 1 Introduction The ability to understand natural language through commonsense reasoning is one of the core focuses in the field of natural language processing. To measure and study the different aspects of com- monsense reasoning, several datasets are devel- oped, such as SocialIQA (Sap et al., 2019b), Com- monsenseQA (Talmor et al., 2018), and Physi- calIQA (Bisk et al., 2020), each requiring different type of commonsense knowledge (e.g., social, taxo- nomic, causal, declarative, etc) to select the correct answer. While large-scale neural systems (Devlin et al., 2018; Yang et al., 2019; Liu et al., 2019b) have shown human-level accuracy on these bench- marks, recent studies (Mitra et al., 2019) also crit- icize that these models solve individual datasets, ∗Work done while at the University of Edinburgh. †Corresponding author rather than learning how to perform general seman- tic reasoning. To this end, Ma et al. (2021) sug- gested zero-shot evaluation as a genuine measure for the reasoning capability of the machine. Inspired by this new metric, in this work, we focus on building unsupervised zero-shot multiple- choice QA systems. That is, we target an arbitrary commonsense reasoning task where conventional approaches (that rely heavily on task-specific super- vision) are not applicable to such zero-shot learning scenarios. To learn QA models without expensive annotation efforts, recent works (Ma et al., 2021; Banerjee and Baral, 2020; Malaviya et al., 2020) propose to generate a synthetic QA dataset using a commonsense KG such as ATOMIC (Sap et al., 2019a) and ConceptNet (Speer et al., 2017). Such an approach mostly focuses only on one spe- cific type of reasoning relations (e.g., if-then re- lation, or declarative relation), neglecting …

Content truncated to 500 words