SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5039 - 5059 May 22-27, 2022 c⃝2022 Association for Computational Linguistics SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu1,2⋆ Brian Lester1 Noah Constant1 Rami Al-Rfou1 Daniel Cer1 Google Research1 University of Massachusetts Amherst2 {ttvu,brianlester,nconstant,rmyeid,cer}@google.com tuvu@cs.umass.edu Abstract There has been growing interest in parameter- efficient methods to apply pre-trained lan- guage models to downstream tasks. Build- ing on the PROMPTTUNING approach of Lester et al. (2021), which learns task-specific soft prompts to condition a frozen pre-trained model to perform different tasks, we propose a novel prompt-based transfer learning approach called SPOT: Soft Prompt Transfer. SPOT first learns a prompt on one or more source tasks and then uses it to initialize the prompt for a target task. We show that SPOT sig- nificantly boosts the performance of PROMPT- TUNING across many tasks. More remarkably, across all model sizes, SPOT matches or out- performs standard MODELTUNING (which fine- tunes all model parameters) on the SUPER- GLUE benchmark, while using up to 27,000× fewer task-specific parameters. To understand where SPOT is most effective, we conduct a large-scale study on task transferability with 26 NLP tasks in 160 combinations, and demon- strate that many tasks can benefit each other via prompt transfer. Finally, we propose an efficient retrieval approach that interprets task prompts as task embeddings to identify similar tasks and predict the most transferable source tasks for a novel target task. 1 Introduction The past few years have seen the rapid develop- ment of ever larger pre-trained language models, where it has repeatedly been shown that scaling up the model size is a key ingredient for achiev- ing the best performance (Devlin et al., 2019; Raf- fel et al., 2020; Brown et al., 2020). While this trend has continued to push the boundaries of pos- sibility across various NLP benchmarks, the sheer size of these models presents a challenge for their practical application. For 100B+ parameter mod- els, fine-tuning and deploying a separate instance ⋆ Work done during an internship at Google Research. 108 109 1010 1011 Model Parameters 50 60 70 80 90 100 SuperGLUE Score P D s (GPT-3) M T P T M - s M T SP T (Ours) Figure 1: Our SPOT approach—which transfers a prompt learned from a mixture of source tasks (here, GLUE) onto target tasks—outperforms vanilla PROMT- TUNING (Lester et al., 2021) and GPT-3 (Brown et al., 2020) on SUPERGLUE by a large margin, matching or outperforming MODELTUNING across all model sizes. At the XXL model size, SPOT even outperforms MULTI- TASKMODELTUNING, which fine-tunes the entire model on the GLUE mixture before fine-tuning it on individ- ual SUPERGLUE tasks. See Appendix A for full results. of the model for each downstream task would be prohibitively expensive. To get around the infeasi- bility of fine-tuning, Brown et al. (2020) propose PROMPTDESIGN, where every downstream task is cast as a language modeling task and the frozen pre- …

Content truncated to 500 words