I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9614–9630 July 9-14, 2023 ©2023 Association for Computational Linguistics I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation Chandra Bhagavatula∗, Jena D. Hwang∗, Doug Downey¶∗, Ronan Le Bras∗, Ximing Lu§∗, Lianhui Qin§, Keisuke Sakaguchi‡, Swabha Swayamdipta†, Peter West§∗, Yejin Choi§∗ ∗Allen Institute for AI, †University of Southern California, ‡Tohoku University, ¶Northwestern University, §University of Washington i2d2.allen.ai Abstract Commonsense capabilities of pre-trained lan- guage models dramatically improve with scale, leading many to believe that scale is the only winning recipe. But is it? Here, we investigate an alternative that a priori seems impossible: can smaller language models (e.g., GPT-2) win over models that are orders of magnitude larger and better (e.g., GPT-3), if powered with novel commonsense distillation algorithms? The key intellectual challenge is to design a learning algorithm that achieves a competitive level of commonsense acquisition, without relying on the benefits of scale. In particular, we study generative models of commonsense knowledge, focusing on the task of generating generics, statements of commonsense facts about every- day concepts, e.g., birds can fly. We introduce I2D2, a novel commonsense dis- tillation framework that loosely follows West et al. (2022)’s Symbolic Knowledge Distilla- tion but breaks the dependence on the extreme- scale teacher model with two innovations: (1) the novel adaptation of NeuroLogic Decoding (Lu et al., 2021) to enhance the generation qual- ity of the weak, off-the-shelf language models, and (2) self-imitation learning to iteratively learn from the model’s own enhanced common- sense acquisition capabilities. Empirical re- sults suggest that scale is not the only way, as novel algorithms can be a promising alternative. Moreover, our study leads to a new corpus of generics, Gen-A-tomic, that is the largest and highest-quality available to date. 1 Introduction Language models (LMs) become better with scale. However, even the largest LMs continue to fail in unexpected ways due to their lack of commonsense (Brachman and Levesque, 2021). Knowledge mod- els – custom LMs trained to generate knowledge— provide on-demand access to task-specific knowl- edge to address this gap (Bosselut et al., 2019). gen-a-tomic Critic Filtering Filter for quality using fine-tuned RoBERTa Self-Imitation Learning Fine-tune LM on its own quality generations Off-the-shelf LM I2D2 Constrained Decoding with Neurologic Generations Generations Generations gen-a-tomic Critic Filtering Filter for quality using fine-tuned RoBERTa Off-the-shelf LM I2D2 Constrained Decoding with Neurologic Generations Generations Generations Self-Imitation Learning Fine-tune LM on its own quality generations Figure 1: I2D2 radically improves over off-the-shelf generation from GPT-2 XL using constrained decoding and self-imitation. Today, the best strategy for training a knowledge model depends on large-scale, albeit noisy knowl- edge generated from a large LM (West et al., 2022). Are massive-scale LMs the only way to build com- monsense capabilities? In addition to being an interesting scientific inquiry, if smaller LMs can in- deed generate high-quality commonsense, training knowledge models will become far more efficient and accessible compared to the state-of-the-art. We study the generation of commonsense knowl- edge …

Content truncated to 500 words