Symbolic Chain-of-Thought Distillation: Small Models Can Also “Think” Step-by-Step

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2665–2679 July 9-14, 2023 ©2023 Association for Computational Linguistics Symbolic Chain-of-Thought Distillation: Small Models Can Also “Think” Step-by-Step Liunian Harold Li∗†, Jack Hessel♣, Youngjae Yu♢, Xiang Ren◦, Kai-Wei Chang† & Yejin Choi♣♡ †University of California, Los Angeles, ♣Allen Institute for Artificial Intelligence ◦University of Southern California, ♢ Yonsei University, ♡University of Washington Abstract Chain-of-thought prompting (e.g., “Let’s think step-by-step") primes large language models to verbalize rationalization for their predic- tions. While chain-of-thought can lead to dra- matic performance gains, benefits appear to emerge only for sufficiently large models (be- yond 50B parameters). We show that orders- of-magnitude smaller models (125M—1.3B parameters) can still benefit from chain-of- thought prompting. To achieve this, we intro- duce Symbolic Chain-of-Thought Distillation (SCoTD), a method to train a smaller student model on rationalizations sampled from a sig- nificantly larger teacher model. Experiments across several commonsense benchmarks show that: 1) SCoTD enhances the performance of the student model in both supervised and few-shot settings, and especially for challenge sets; 2) sampling many reasoning chains per instance from the teacher is paramount; and 3) after distillation, student chain-of-thoughts are judged by humans as comparable to the teacher, despite orders of magnitude fewer pa- rameters. We test several hypotheses regarding what properties of chain-of-thought samples are important, e.g., diversity vs. teacher like- lihood vs. open-endedness. We release our corpus of chain-of-thought samples and code. 1 Introduction Empirical scaling laws suggest that the accuracy of Large Language Models (LLMs) on benchmark tasks can be improved by increasing model size and pre-training data volume (Hoffmann et al., 2022). Beyond these training-time improvements, how- ever, an inference-time strategy dubbed “chain-of- thought" (CoT) prompting,1 i.e., eliciting verbaliza- tions of predictive processes via key-phrases like “Let’s think step-by-step" (Kojima et al., 2022), can *Work done during an internship at AI2. 1Sometimes called “self-rationalization" or “prompting with explanations.” We will use these terms interchangeably in this paper. 🤖 Teacher Model (GPT-3, 175B) Where is a well-used toy car likely to be found? Let’s think step by step… Student Model + Symbolic Chain-of-thought Distillation 🤖 🤖 Student Model (OPT, 125M-1.3B) A used car dealership… Because you can’t buy a… If the toy car is used, then… The owner of the toy car… N samples (+Filters) A used toy can usually be found in the home of a… Figure 1: Symbolic Chain-of-thought Distillation (SCoTD) applied to a student model, ranging in size from 125M–1.3B parameters. We show that fine-tuning on a (diverse and potentially filtered) corpus of expert chain-of-thought demonstrations from a teacher model is an effective strategy to make smaller models capable of chain-of-thought reasoning. similarly improve performance, e.g., Suzgun et al. (2022) demonstrate additional performance gains on a hard subset of the BigBench tasks (BIG-bench collaboration, 2022) using chain-of-thought. However, chain-of-thought prompting has only been shown to be beneficial for models of suffi- cient scale (e.g., with more than 60B parameters (Wei et al., 2022b)). In this …

Content truncated to 500 words