Reliable Gradient-free and Likelihood-free Prompt Tuning

Findings of the Association for Computational Linguistics: EACL 2023, pages 2416–2429 May 2-6, 2023 ©2023 Association for Computational Linguistics Reliable Gradient-free and Likelihood-free Prompt Tuning Maohao Shen1, Soumya Ghosh3, Prasanna Sattigeri3, Subhro Das3, Yuheng Bu2, Gregory Wornell1 1 Massachusetts Institute of Technology 2 University of Florida 3 MIT-IBM Watson AI Lab, IBM Research Abstract Due to privacy or commercial constraints, large pre-trained language models (PLMs) are often offered as black-box APIs. Fine-tuning such models to downstream tasks is challenging be- cause one can neither access the model’s in- ternal representations nor propagate gradients through it. This paper addresses these chal- lenges by developing techniques for adapting PLMs with only API access. Building on re- cent work on soft prompt tuning, we develop methods to tune the soft prompts without requir- ing gradient computation. Further, we develop extensions that in addition to not requiring gra- dients also do not need to access any internal representation of the PLM beyond the input embeddings. Moreover, instead of learning a single prompt, our methods learn a distribution over prompts allowing us to quantify predic- tive uncertainty. Ours is the first work to con- sider uncertainty in prompts when only having API access to the PLM. Finally, through exten- sive experiments, we carefully vet the proposed methods and find them competitive with (and sometimes even improving on) gradient-based approaches with full access to the PLM. 1 Introduction Pre-trained language models (PLMs) are versa- tile learners and demonstrate impressive few-shot capabilities (Brown et al., 2020) and promising performance (Radford et al., 2018; Devlin et al., 2018; Raffel et al., 2020; Lewis et al., 2019) on various downstream tasks such as text classifica- tion (Kowsari et al., 2019), commonsense reason- ing (Zellers et al., 2018), question answering (Ra- jpurkar et al., 2016), and machine translation (Bah- danau et al., 2014). The conventional approach to adapting PLMs to downstream tasks involves fine-tuning the model (Peters et al., 2018; Devlin et al., 2018). Although fine-tuning is effective, it can be chal- lenging to do in practice. First, fine-tuning large language models are compute and memory inten- sive, e.g., a large model like GPT-3 (Brown et al., 2020) contains billions of parameters. Further, it is inefficient to adapt a PLM to a large number of downstream tasks since each task would require storing a copy of model parameters. Prompt tuning alleviates these issues by provid- ing an efficient way to adapt a PLM to a down- stream task. It only learns a small number of prompt parameters while keeping the large PLM frozen but still achieves comparable performance to fine-tuning the entire PLM (Liu et al., 2021a; Shin et al., 2020; Lester et al., 2021; Liu et al., 2021c). Although more efficient than traditional fine- tuning, prompt tuning still requires the propagation of gradients through the entire PLM. Beyond be- ing computationally expensive, this may not be possible due to privacy risks or legal and com- mercial constraints. In fact, large PLMs are of- ten only made available in the …

Content truncated to 500 words