SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2177–2190 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2177 SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization Haoming Jiang ∗ Georgia Tech jianghm@gatech.edu Pengcheng He, Weizhu Chen Microsoft Dynamics 365 AI {penhe,wzchen}@microsoft.com Xiaodong Liu, Jianfeng Gao Microsoft Research {xiaodl,jfgao}@microsoft.com Tuo Zhao Georgia Tech tourzhao@gatech.edu Abstract Transfer learning has fundamentally changed the landscape of natural language processing (NLP). Many state-of-the-art models are first pre-trained on a large text corpus and then fine-tuned on downstream tasks. However, due to limited data resources from downstream tasks and the extremely high complexity of pre-trained models, aggressive fine-tuning of- ten causes the fine-tuned model to overfit the training data of downstream tasks and fail to generalize to unseen data. To address such an issue in a principled manner, we propose a new learning framework for robust and efficient fine-tuning for pre-trained models to attain better generalization performance. The pro- posed framework contains two important in- gredients: 1. Smoothness-inducing regulariza- tion, which effectively manages the complex- ity of the model; 2. Bregman proximal point optimization, which is an instance of trust- region methods and can prevent aggressive up- dating. Our experiments show that the pro- posed framework achieves new state-of-the-art performance on a number of NLP tasks includ- ing GLUE, SNLI, SciTail and ANLI. More- over, it also outperforms the state-of-the-art T5 model, which is the largest pre-trained model containing 11 billion parameters, on GLUE. 1 1 Introduction The success of natural language processing (NLP) techniques relies on huge amounts of labeled data in many applications. However, large amounts of labeled data are usually prohibitive or expensive to obtain. To address this issue, researchers have resorted to transfer learning. Transfer learning considers the scenario, where we have limited labeled data from the target do- main for a certain task, but we have relevant tasks ∗ Work was done during an internship at Microsoft Dy- namics 365 AI. 1https://github.com/namisan/mt-dnn with a large amount of data from different domains (also known as out-of-domain data). The goal is to transfer the knowledge from the high-resource domains to the low-resource target domain. Here we are particularly interested in the popular two- stage transfer learning framework (Pan and Yang, 2009). The first stage is pre-training, where a high-capacity model is trained for the out-of- domain high-resource relevant tasks. The sec- ond stage is fine-tuning, where the high-capacity model is adapted to the low-resource task in the target domain. For many applications in NLP, most popular transfer learning methods choose to pre-train a large language model, e.g., ELMo (Peters et al., 2018), GPT (Radford et al., 2019) and BERT (De- vlin et al., 2019). Such a language model can cap- ture general semantic and syntactic information that can be further used in downstream NLP tasks. The language model is particularly attractive, be- cause it can be trained in a completely unsuper- vised manner with huge amount of unlabeled data, which …

Content truncated to 500 words