Prefix-Tuning: Optimizing Continuous Prompts for Generation

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4582–4597 August 1–6, 2021. ©2021 Association for Computational Linguistics 4582 Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Stanford University xlisali@stanford.edu Percy Liang Stanford University pliang@cs.stanford.edu Abstract Fine-tuning is the de facto way of leveraging large pretrained language models for down- stream tasks. However, fine-tuning modifies all the language model parameters and there- fore necessitates storing a full copy for each task. In this paper, we propose prefix-tuning, a lightweight alternative to fine-tuning for natu- ral language generation tasks, which keeps lan- guage model parameters frozen and instead op- timizes a sequence of continuous task-specific vectors, which we call the prefix. Prefix-tuning draws inspiration from prompting for language models, allowing subsequent tokens to attend to this prefix as if it were “virtual tokens”. We apply prefix-tuning to GPT-2 for table-to- text generation and to BART for summariza- tion. We show that by modifying only 0.1% of the parameters, prefix-tuning obtains compara- ble performance in the full data setting, outper- forms fine-tuning in low-data settings, and ex- trapolates better to examples with topics that are unseen during training. 1 Introduction Fine-tuning is the prevalent paradigm for using large pretrained language models (LMs) (Radford et al., 2019; Devlin et al., 2019) to perform down- stream tasks (e.g., summarization), but it requires updating and storing all the parameters of the LM. Consequently, to build and deploy NLP systems that rely on large pretrained LMs, one currently needs to store a modified copy of all the LM pa- rameters for each task. This can be prohibitively expensive given the size of current LMs; for exam- ple, GPT-2 has 774M parameters (Radford et al., 2019) and GPT-3 has 175B parameters (Brown et al., 2020). A natural approach to this problem is lightweight fine-tuning, which freezes most of the pretrained parameters and only tunes a smaller set of param- eters. For example, adapter-tuning (Rebuffi et al., Figure 1: Fine-tuning (top) updates all LM param- eters (the red Transformer box) and requires storing a full model copy for each task. We propose prefix- tuning (bottom), which freezes the LM parameters and only optimizes the prefix (the red prefix blocks). Con- sequently, we only need to store the prefix for each task, making prefix-tuning modular and space-efficient. Note that each vertical block denote transformer activa- tions at one time step. 2017; Houlsby et al., 2019) inserts additional task- specific layers between the layers of pretrained language models. Adapter-tuning has promising performance on natural language understanding and generation benchmarks, attaining comparable performance with fine-tuning while adding only around 2–4% task-specific parameters (Houlsby et al., 2019; Lin et al., 2020). At the limit, GPT-3 (Brown et al., 2020) can be deployed using in-context learning, which is a form of prompting, without modifying any LM parameters. In in-context learning, Brown et al. (2020) prepend a natural language task instruction (e.g., TL;DR for summarization) and a few exam- …

Content truncated to 500 words