The Power of Scale for Parameter-Efficient Prompt Tuning

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3045 The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester∗ Rami Al-Rfou Noah Constant Google Research {brianlester,rmyeid,nconstant}@google.com Abstract In this work, we explore “prompt tuning,” a simple yet effective mechanism for learn- ing “soft prompts” to condition frozen lan- guage models to perform specific downstream tasks. Unlike the discrete text prompts used by GPT-3, soft prompts are learned through back- propagation and can be tuned to incorporate signals from any number of labeled examples. Our end-to-end learned approach outperforms GPT-3’s few-shot learning by a large margin. More remarkably, through ablations on model size using T5, we show that prompt tuning be- comes more competitive with scale: as mod- els exceed billions of parameters, our method “closes the gap” and matches the strong per- formance of model tuning (where all model weights are tuned). This finding is especially relevant because large models are costly to share and serve and the ability to reuse one frozen model for multiple downstream tasks can ease this burden. Our method can be seen as a simplification of the recently proposed “prefix tuning” of Li and Liang (2021) and we provide a comparison to this and other similar approaches. Finally, we show that condition- ing a frozen model with soft prompts confers benefits in robustness to domain transfer and enables efficient “prompt ensembling.” We release code and model checkpoints to repro- duce our experiments.1 1 Introduction With the wide success of pre-trained large lan- guage models, a range of techniques has arisen to adapt these general-purpose models to downstream tasks. ELMo (Peters et al., 2018) proposed freezing the pre-trained model and learning a task-specific weighting of its per-layer representations. How- ever, since GPT (Radford et al., 2018) and BERT ∗Work done as a Google AI Resident. 1https://github.com/google-research/ prompt-tuning 108 109 1010 1011 Model Parameters 50 60 70 80 90 100 SuperGLUE Score Model Tuning Model Tuning (Multi-task) Prompt Design Prompt Tuning Figure 1: Standard model tuning of T5 achieves strong performance, but requires storing separate copies of the model for each end task. Our prompt tuning of T5 matches the quality of model tuning as size increases, while enabling the reuse of a single frozen model for all tasks. Our approach significantly outperforms few- shot prompt design using GPT-3. We show mean and standard deviation across 3 runs for tuning methods. (Devlin et al., 2019), the dominant adaptation tech- nique has been model tuning (or “fine-tuning”), where all model parameters are tuned during adap- tation, as proposed by Howard and Ruder (2018). More recently, Brown et al. (2020) showed that prompt design (or “priming”) is surprisingly effec- tive at modulating a frozen GPT-3 model’s behavior through text prompts. Prompts are typically com- posed of a task description and/or several canonical examples. This return to “freezing” pre-trained models is appealing, especially as model size con- tinues to increase. Rather than requiring a separate copy …

Content truncated to 500 words