Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7870–7881, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 7870 Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting Sanyuan Chen1, Yutai Hou1, Yiming Cui1,2, Wanxiang Che1, Ting Liu1, Xiangzhan Yu1 1School of Computer Science and Technology, Harbin Institute of Technology, China 2Joint Laboratory of HIT and iFLYTEK Research (HFL), Beijing, China {sychen, ythou, ymcui, car, tliu}@ir.hit.edu.cn, yxz@hit.edu.cn Abstract Deep pretrained language models have achieved great success in the way of pretrain- ing first and then fine-tuning. But such a sequential transfer learning paradigm often confronts the catastrophic forgetting problem and leads to sub-optimal performance. To fine-tune with less forgetting, we propose a recall and learn mechanism, which adopts the idea of multi-task learning and jointly learns pretraining tasks and downstream tasks. Specifically, we introduce a Pretraining Sim- ulation mechanism to recall the knowledge from pretraining tasks without data, and an Objective Shifting mechanism to focus the learning on downstream tasks gradually. Experiments show that our method achieves state-of-the-art performance on the GLUE benchmark. Our method also enables BERT- base to achieve better average performance than directly fine-tuning of BERT-large. Fur- ther, we provide the open-source RECADAM optimizer, which integrates the proposed mechanisms into Adam optimizer, to facility the NLP community.1 1 Introduction Deep Pretrained Language Models (LMs), such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2019), have significantly altered the land- scape of Natural Language Processing (NLP), and a wide range of NLP tasks has been promoted by these pretrained language models. These successes are mainly achieved through Sequential Transfer Learning (Ruder, 2019): pretrain a language model on large-scale unlabeled data and then adapt it to downstream tasks. The adaptation step is usually conducted in two manners: fine-tuning or freez- ing pretrained weights. In practice, fine-tuning is adopted more widely due to its flexibility (Phang et al., 2018; Peters et al., 2019; Lan et al., 2020). 1https://github.com/Sanyuan-Chen/RecAdam Despite the great success, sequential transfer learning of deep pretrained LMs is prone to catas- trophic forgetting during the adaptation step. Catas- trophic forgetting is a common problem for se- quential transfer learning, and it happens when a model forgets previously learned knowledge and overfits to target domains (McCloskey and Co- hen, 1989; Kirkpatrick et al., 2017). To remedy the catastrophic forgetting in transferring deep pre- trained LMs, existing efforts mainly explore fine- tuning tricks to forget less. ULMFiT (Howard and Ruder, 2018) introduced discriminative fine-tuning, slanted triangular learning rates, and gradual un- freezing for LMs fine-tuning. Lee et al. (2020) reduced forgetting in BERT fine-tuning by ran- domly mixing pretrained parameters to a down- stream model in a dropout-style. Instead of learning pretraining tasks and down- stream tasks in sequence, Multi-task Learning learns both of them simultaneously, thus can in- herently avoid the catastrophic forgetting prob- lem. Xue et al. (2019) tackled forgetting in au- tomatic speech recognition by jointly training the model with previous and target tasks. Kirkpatrick et al. (2017) …

Content truncated to 500 words