Improving Stability of Fine-Tuning Pretrained Language Models via Component-Wise Gradient Norm Clipping

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4854 - 4859 December 7-11, 2022 ©2022 Association for Computational Linguistics Improving Stability of Fine-Tuning Pretrained Language Models via Component-Wise Gradient Norm Clipping Chenghao Yang1, Xuezhe Ma2 1University of Chicago 2University of Southern California yangalan1996@gmail.com, xuezhema@isi.edu Abstract Fine-tuning over large pretrained language models (PLMs) has established many state- of-the-art results. Despite its superior perfor- mance, such fine-tuning can be unstable, re- sulting in significant variance in performance and potential risks for practical applications. Previous works have attributed such instabil- ity to the catastrophic forgetting problem in the top layers of PLMs, which indicates itera- tively fine-tuning layers in top-down manner is a promising solution. In this paper, we first point out that this method does not always work out due to different convergence speeds of different layers/modules. Inspired by this observation, we propose a simple component- wise gradient norm clipping method to adjust the convergence speed for different compo- nents. Experiment results demonstrate that our method achieves consistent improvements in terms of generalization performance, conver- gence speed and training stability. The code- base can be found at https://github.com/ yangalan123/FineTuningStability. 1 Introduction Fine-tuning over large pretrained language models (PLMs), which achieved remarkable performance over various benchmarks, has become the de facto paradigm for several current natural language pro- cessing (NLP) systems. However, fine-tuning can be unstable in terms of significant variance in met- rics, resulting in even worse-than-random failed models (Devlin et al., 2019; Lee et al., 2019; Dodge et al., 2020; Mosbach et al., 2020). Catastrophic forgetting (Kirkpatrick et al., 2017) during fine-tuning of PLMs is one common expla- nation for this instability (Lee et al., 2019), i.e., PLMs may lose their rich domain-agnostic knowl- edge acquired by language model pretraining in the process of fine-tuning. Through layer-replacement experiments between pretrained models and fine- tuned models, Mosbach et al. (2020) further con- 0 5 10 15 20 25 #(Iterations) 50 55 60 65 70 75 Accuracy RTE Fine-Tuning Performance Change Across Time Devlin et al., 2019 baseline Devlin et al., 2019 baseline + bias correction CWGNC (ours) Gradual Unfreezing Gradual Unfreezing (Restart) Figure 1: Fine-tuning performance over time on RTE datasets. Here we fine-tune over BERT-large-uncased model (Devlin et al., 2019). In each iteration, we train the model for 3 epochs and the errorbar is plotted based on 5 different runs. nected the catastrophic forgetting problem to the optimization problem on top layers. These findings give rise to a straightforward way to enhance the fine-tuning stability: how about fine- tuning the model from top to bottom to reduce the parameter changes and hence mitigate the catas- trophic forgetting problem? This is reminiscent of the gradual unfreezing (Howard and Ruder, 2018), which does layer-wise top-down fine-tuning and unfreezing new layers only when the layers above have been fine-tuned. Therefore, the newly un- frozen layer would only be tuned for a slightly eas- ier optimization problem at each iteration, leading to much fewer changes to the parameters. However, based …

Content truncated to 500 words