Hard Sample Aware Prompt-Tuning

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 12356–12369 July 9-14, 2023 ©2023 Association for Computational Linguistics Hard Sample Aware Prompt-Tuning Yuanjian Xu1,2∗ † , Qi An1,2∗ † , Jiahuan Zhang1∗, Peng Li1 , Zaiqing Nie1,3‡ Institute for AI Industry Research (AIR), Tsinghua University1 School of Software and Microelectronics, Peking University2 Beijing Academy of Artificial Intelligence3 {xyj, angel}@stu.pku.edu.cn; {zhangjiahuan, lipeng, zaiqing}@air.tsinghua.edu.cn Abstract Prompt-tuning based few-shot learning has gar- nered increasing attention in recent years due to its efficiency and promising capability. To achieve the best performance for natural lan- guage processing (NLP) tasks with just a few samples, it is vital to include as many informa- tive samples as possible and to avoid mislead- ing ones. However, there is no work in prompt- tuning literature addressing the problem of dif- ferentiating informative hard samples from mis- leading ones in model training, which is chal- lenging due to the lack of supervision signals about the quality of the samples to train a well- performed model. We propose a framework named Hard Sample Aware Prompt-Tuning (HardPT) to solve the non-differentiable prob- lem in hard sample identification with rein- forcement learning, and to strengthen the dis- crimination of the feature space without chang- ing the original data distribution via an adaptive contrastive learning method. An extensive em- pirical study on a series of NLP tasks demon- strates the capability of HardPT in few-shot scenarios. HardPT obtains new state-of-the-art results on all evaluated NLP tasks, including pushing the SST-5 accuracy to 49.5% (1.1% point absolute improvement), QNLI accuracy to 74.6% (1.9% absolute improvement), NMLI accuracy to 71.5 (0.7% absolute improvement), TACREV F1-score to 28.2 (1.0 absolute im- provement), and i2b2/VA F1-score to 41.2 (1.3 absolute improvement). 1 Introduction In recent years, self-supervised pre-trained lan- guage models (PLMs) like GPT (Radford et al., 2019), and BERT (Devlin et al., 2019) have gained significant popularity in various natural language ∗ Equal contribution. † Work done during Yuanjian Xu and Qi An’s internship at AIR, Tsinghua University. ‡ Corresponding author. processing (NLP) tasks. These PLMs follow a general paradigm of transferring contextual knowl- edge to specific NLP tasks by fine-tuning model parameters. With the release of GPT-3 (Brown et al., 2020), prompt-tuning has received much at- tention of late for its outstanding performance in downstream NLP tasks, without the cost of fine- tuning giant PLMs. Particularly, prompt-tuning has shown great superiority in few-shot learning scenarios over fine-tuning methods. To achieve the best performance for NLP tasks with just a few samples, it is vital to include as many informative samples as possible and to avoid misleading ones. However, there is no work in the literature addressing the problem of differentiating informative hard samples from misleading ones in prompt-tuning. The concept of hard samples has been discussed in the field of computer vision for object detection and image classification tasks (Shrivastava et al., 2016; Lin et al., 2017). During the training process, high-loss samples are typically selected as hard …

Content truncated to 500 words