BBTv2: Towards a Gradient-Free Future with Large Language Models

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3916 - 3930 December 7-11, 2022 ©2022 Association for Computational Linguistics BBTv2: Towards a Gradient-Free Future with Large Language Models Tianxiang Sun♢♡ Zhengfu He♢ Hong Qian♠ Yunhua Zhou♢♡ Xuanjing Huang♢♡ Xipeng Qiu♢♡∗ ♢School of Computer Science, Fudan University ♡Shanghai Key Laboratory of Intelligent Information Processing, Fudan University ♠School of Computer Science and Technology, East China Normal University {txsun19,zfhe19,zhouyh20,xjhuang,xpqiu}@fudan.edu.cn hqian@cs.ecnu.edu.cn Abstract Most downstream adaptation methods tune all or part of the parameters of pre-trained models (PTMs) through gradient descent, where the tuning cost increases linearly with the growth of the model size. By contrast, gradient-free methods only require the forward computa- tion of the PTM to tune the prompt, retain- ing the benefits of efficient tuning and deploy- ment. Though, past work on gradient-free tun- ing often introduces gradient descent to seek a good initialization of prompt and lacks versa- tility across tasks and PTMs. In this paper, we present BBTv2, an improved version of Black- Box Tuning (Sun et al., 2022b), to drive PTMs for few-shot learning. We prepend continuous prompts to every layer of the PTM and propose a divide-and-conquer gradient-free algorithm to optimize the prompts at different layers al- ternately. Extensive experiments across var- ious tasks and PTMs show that BBTv2 can achieve comparable performance to full model tuning and state-of-the-art parameter-efficient methods (e.g., Adapter, LoRA, BitFit, etc.) un- der few-shot settings while maintaining much fewer tunable parameters. 1 Introduction The past few years have witnessed remarkable progress of large language models (LLMs) (Devlin et al., 2019; Raffel et al., 2020; Brown et al., 2020). It has been repeatedly demonstrated that scaling up the model size is promising to achieve better performance. However, the growing model size also leads to a linear increase in tuning cost. Fine- tuning and deploying a separate copy of the LLM for each downstream task become prohibitively ex- pensive. To that end, much effort has been devoted to parameter-efficient tuning (PET) (He et al., 2021; Ding et al., 2022), which only tunes a small portion of parameters while keeping most of the parame- ters of the LLM unchanged. By PET, LLMs can ∗ Corresponding author. Figure 1: BBTv2 achieves comparable results to gradient-based methods on average performance over 7 language understanding tasks (§5.1) with much fewer tunable parameters. Size of the circle is proportional to the standard deviation of the performance. All the methods are evaluated on RoBERTaLARGE. be specialized to a downstream task at inference time by activating a small number of task-specific parameters. Though it is deployment-efficient, tun- ing the small portion of parameters still requires back-propagation through the entire LLM, which is expensive or even infeasible for many practitioners. To make LLMs benefit a wider range of au- diences, a common practice is to release LLMs as a service and allow users to access the pow- erful LLMs through their inference APIs (Brown et al., 2020). In such a scenario, called Languaged- Model-as-a-Service (LMaaS) (Sun et al., …

Content truncated to 500 words