Cold-Start Data Selection for Better Few-shot Language Model Fine-tuning: A Prompt-based Uncertainty Propagation Approach

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2499–2521 July 9-14, 2023 ©2023 Association for Computational Linguistics Cold-Start Data Selection for Better Few-shot Language Model Fine-tuning: A Prompt-based Uncertainty Propagation Approach Yue Yu1 Rongzhi Zhang1 Ran Xu2 Jieyu Zhang3 Jiaming Shen4 Chao Zhang1 1 Georgia Institute of Technology 3 University of Washington2 Emory University 4 Google {yueyu, rongzhi.zhang, chaozhang}@gatech.edu, {ran.xu}@emory.edu, jieyuz2@cs.washington.edu, jmshen@google.com Abstract Large Language Models have demonstrated re- markable few-shot performance, but the per- formance can be sensitive to the selection of few-shot instances. We present PATRON, a prompt-based data selection method for pre- trained language model fine-tuning under cold- start scenarios, i.e., no initial labeled data are available. In PATRON, we design (1) a prompt- based uncertainty propagation approach to es- timate the importance of data points and (2) a partition-then-rewrite (PTR) strategy to pro- mote sample diversity when querying for anno- tations. Experiments on six text classification datasets show that PATRON outperforms the strongest cold-start data selection baselines by up to 6.9%. Besides, with 128 labels only, PA- TRON achieves 91.0% and 92.1% of the fully supervised performance based on vanilla fine- tuning and prompt-based learning respectively. Our implementation of PATRON is available at https://github.com/yueyu1030/Patron. 1 Introduction Pre-trained language models (PLMs) (Devlin et al., 2019; Liu et al., 2019; Raffel et al., 2020) have achieved competitive performance with limited la- beled data (Gao et al., 2021a; Schick and Schütze, 2021a,b) for many natural language processing (NLP) tasks. However, there still exists a non- negligible gap between the performance of few- shot and fully-supervised PLMs. Besides, when the task-specific data for fine-tuning is small, the per- formance of PLMs can have high variance (Bragg et al., 2021). As illustrated in Figure 1, when fine-tuning RoBERTa-base (Liu et al., 2019) on different subsets of AG News dataset with 32 la- bels, the performance on the test set varies up to 10% for vanilla fine-tuning and 5% for prompt- based learning (Gao et al., 2021a). Such large vari- ations demonstrate the crucial need for strategical selection of training data to improve PLMs’ perfor- mance under low-data regimes. 65.0 70.0 75.0 80.0 85.0 Accuracy (%) Seed1 Seed2 Seed3 Seed4 Seed5 PATRON Splits (a) Fine-tuning 77.5 80.0 82.5 85.0 87.5 Accuracy (%) Seed1 Seed2 Seed3 Seed4 Seed5 PATRON Splits (b) Prompt-based Learning Figure 1: The performance with large variances of vanilla fine-tuning and prompt-based learning on 5 ran- dom samplings, compared with better performance with low variances of PATRON (our proposed selection strat- egy) on AG News (Zhang et al., 2015) with 32 labels. To solicit training data intelligently, active learn- ing (AL) (Settles, 2011) has been proposed to adap- tively annotate unlabeled data (Ash et al., 2020; Ein-Dor et al., 2020; Zhang and Plank, 2021; Mar- gatina et al., 2021, 2022). Despite their efficacy, most of these works assume there are hundreds, or even thousands of labels in the initial stage, and query similarly significant amounts of labeled data in each AL …

Content truncated to 500 words