AcTune: Uncertainty-Based Active Self-Training for Active Fine-Tuning of Pretrained Language Models

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1422 - 1436 July 10-15, 2022 ©2022 Association for Computational Linguistics ACTUNE: Uncertainty-Based Active Self-Training for Active Fine-Tuning of Pretrained Language Models Yue Yu1, Lingkai Kong1, Jieyu Zhang2, Rongzhi Zhang1, Chao Zhang1 1Georgia Institute of Technology, Atlanta, GA 2University of Washington, Seattle, WA {yueyu, lkkong, rongzhi.zhang, chaozhang}@gatech.edu, jieyuz2@cs.washington.edu Abstract While pre-trained language model (PLM) fine- tuning has achieved strong performance in many NLP tasks, the fine-tuning stage can be still demanding in labeled data. Recent works have resorted to active fine-tuning to improve the label efficiency of PLM fine-tuning, but none of them investigate the potential of un- labeled data. We propose ACTUNE, a new framework that leverages unlabeled data to improve the label efficiency of active PLM fine-tuning. ACTUNE switches between data annotation and model self-training based on uncertainty: it selects high-uncertainty unla- beled samples for active annotation and low- uncertainty ones for model self-training. Un- der this framework, we design (1) a region- aware sampling strategy that reduces redun- dancy when actively querying for annotations and (2) a momentum-based memory bank that dynamically aggregates the model’s pseudo la- bels to suppress label noise in self-training. Experiments on 6 text classification datasets show that ACTUNE outperforms the strongest active learning and self-training baselines and improves the label efficiency of PLM fine- tuning by 56.2% on average. Our imple- mentation is available at https://github. com/yueyu1030/actune. 1 Introduction Fine-tuning pre-trained language models (PLMs) has achieved much success in natural language processing (NLP) (Devlin et al., 2019; Liu et al., 2019; Brown et al., 2020). One benefit of PLM fine-tuning is the promising performance it offers when consuming only a few labeled data (Bansal et al., 2020; Gao et al., 2021). However, there are still significant gaps between few-shot and fully-supervised PLM fine-tuning in many tasks. Besides, the performance of few-shot PLM fine- tuning can be sensitive to different sets of training data (Bragg et al., 2021). Therefore, there is a crucial need for approaches that make PLM fine- tuning more label-efficient and robust to selection of training data, especially for applications where labeled data are scarce and expensive to obtain. Towards this goal, researchers have recently re- sorted to active fine-tuning of PLMs and achieved comparable performance to fully-supervised meth- ods with much less annotated samples (Ein-Dor et al., 2020; Margatina et al., 2021a,b; Yuan et al., 2020). Nevertheless, they usually neglect unlabeled data, which can be useful for improving label effi- ciency for PLM fine-tuning (Du et al., 2021). To incorporate unlabeled data into active learning, ef- forts have been made in the semi-supervised active learning literature (Wang et al., 2016; Rottmann et al., 2018; Siméoni et al., 2020). However, the query strategies proposed in these works can return highly redundant samples due to limited representa- tion power, resulting in suboptimal label efficiency. Moreover, they usually rely on pseudo-labeling to utilize unlabeled data, which requires greater (yet often absent) care …

Content truncated to 500 words