GraDA: Graph Generative Data Augmentation for Commonsense Reasoning

Proceedings of the 29th International Conference on Computational Linguistics, pages 4499–4516 October 12–17, 2022. 4499 GRADA: Graph Generative Data Augmentation for Commonsense Reasoning Adyasha Maharana Mohit Bansal Department of Computer Science University of North Carolina at Chapel Hill {adyasha,mbansal}@cs.unc.edu Abstract Recent advances in commonsense reasoning have been fueled by the availability of large- scale human annotated datasets. Manual an- notation of such datasets, many of which are based on existing knowledge bases, is expen- sive and not scalable. Moreover, it is challeng- ing to build augmentation data for common- sense reasoning because the synthetic questions need to adhere to real-world scenarios. Hence, we present GRADA, a graph-generative data augmentation framework to synthesize factual data samples from knowledge graphs for com- monsense reasoning datasets. First, we train a graph-to-text model for conditional genera- tion of questions from graph entities and rela- tions. Then, we train a generator with GAN loss to generate distractors for synthetic ques- tions. Our approach improves performance for SocialIQA, CODAH, HellaSwag and Common- senseQA, and works well for generative tasks like ProtoQA. We show improvement in robust- ness to semantic adversaries after training with GRADA and provide human evaluation of the quality of synthetic datasets in terms of fac- tuality and answerability. Our work provides evidence and encourages future research into graph-based generative data augmentation. 1 1 Introduction Recent work has seen the emergence of several datasets for improving commonsense reasoning of language models through tasks like question an- swering (QA) (Sap et al., 2019b; Talmor et al., 2019; Bisk et al., 2020) and natural language infer- ence (Bhagavatula et al., 2020; Zellers et al., 2019; Sakaguchi et al., 2020). Some of these datasets are based on existing knowledge graphs that rep- resent different aspects of commonsense through entities and relations. For example, annotators for SocialIQA (Sap et al., 2019b) were shown an event 1Code and synthetic data files are available at https: //github.com/adymaharana/GraDA. Original Dataset Question Generator Adversarial Choice Generator Discriminator Questions Choices Synthetic Dataset Knowledge Graph Filtering Learner Model OptionGAN GraphGPT2 Figure 1: GRADA framework: The original dataset is used to train GraphGPT2, a graph-to-text question generator and OptionGAN, a distractor generator. The synthetic dataset is subjected to filtering and used to train the model in combniation with the original dataset. from the inferential knowledge graph ATOMIC (Sap et al., 2019a) and instructed to turn it into a sentence by adding names, filling placeholders and adding context, etc. For multiple-choice QA datasets, annotators are also instructed to write dis- tractor choices for each question. These useful datasets are collected through a time-taking and money-intensive crowdsourcing process which is hard to scale. Large pretrained models like GPT2 (Radford et al., 2018) can be finetuned to generate sentences from narrow data distributions, and it has recently been leveraged to augment datasets for text classification (Anaby-Tavor et al., 2020) and question answering (Puri et al., 2020; Yang et al., 2020). However, it is challenging to gener- ate augmentation data for commonsense reasoning because the generated questions and answers …

Content truncated to 500 words