Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1063–1077 June 6–11, 2021. ©2021 Association for Computational Linguistics 1063 Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach Yue Yu∗ Simiao Zuo∗ Haoming Jiang Wendi Ren Tuo Zhao Chao Zhang Georgia Institute of Technology, Atlanta, GA, USA {yueyu,simiaozuo,jianghm,wren44,tourzhao,chaozhang}@gatech.edu Abstract Fine-tuned pre-trained language models (LMs) have achieved enormous success in many nat- ural language processing (NLP) tasks, but they still require excessive labeled data in the fine- tuning stage. We study the problem of fine- tuning pre-trained LMs using only weak super- vision, without any labeled data. This prob- lem is challenging because the high capacity of LMs makes them prone to overfitting the noisy labels generated by weak supervision. To address this problem, we develop a con- trastive self-training framework, COSINE, to enable fine-tuning LMs with weak supervision. Underpinned by contrastive regularization and confidence-based reweighting, our framework gradually improves model fitting while effec- tively suppressing error propagation. Experi- ments on sequence, token, and sentence pair classification tasks show that our model outper- forms the strongest baseline by large margins and achieves competitive performance with fully-supervised fine-tuning methods. Our implementation is available on https:// github.com/yueyu1030/COSINE. 1 Introduction Language model (LM) pre-training and fine-tuning achieve state-of-the-art performance in various nat- ural language processing tasks (Peters et al., 2018; Devlin et al., 2019; Liu et al., 2019; Raffel et al., 2019). Such approaches stack task-specific layers on top of pre-trained language models, e.g., BERT (Devlin et al., 2019), then fine-tune the models with task-specific data. During fine-tuning, the semantic and syntactic knowledge in the pre-trained LMs is adapted for the target task. Despite their success, one bottleneck for fine-tuning LMs is the require- ment of labeled data. When labeled data are scarce, the fine-tuned models often suffer from degraded performance, and the large number of parameters can cause severe overfitting (Xie et al., 2019). ∗Equal Contribution. To relieve the label scarcity bottleneck, we fine- tune the pre-trained language models with only weak supervision. While collecting large amounts of clean labeled data is expensive for many NLP tasks, it is often cheap to obtain weakly labeled data from various weak supervision sources, such as semantic rules (Awasthi et al., 2020). For ex- ample, in sentiment analysis, we can use rules ‘terrible’→Negative (a keyword rule) and ‘* not recommend *’→Negative (a pat- tern rule) to generate large amounts of weak labels. Fine-tuning language models with weak supervi- sion is nontrivial. Excessive label noise, e.g., wrong labels, and limited label coverage are common and inevitable in weak supervision. Although existing fine-tuning approaches (Xu et al., 2020; Zhu et al., 2020; Jiang et al., 2020) improve LMs’ generaliza- tion ability, they are not designed for noisy data and are still easy to overfit on the noise. Moreover, ex- isting works on tackling label noise are flawed and are not designed for fine-tuning LMs. For exam- ple, Ratner et al. (2020); Varma and Ré (2018) …

Content truncated to 500 words