Investigating Learning Dynamics of BERT Fine-Tuning

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 87–92 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 87 Investigating Learning Dynamics of BERT Fine-Tuning Yaru Hao†∗, Li Dong‡, Furu Wei‡, Ke Xu† †Beihang University ‡Microsoft Research {haoyaru@,kexu@nlsde.}buaa.edu.cn {lidong1,fuwei}@microsoft.com Abstract The recently introduced pre-trained language model BERT advances the state-of-the-art on many NLP tasks through the fine-tuning ap- proach, but few studies investigate how the fine-tuning process improves the model per- formance on downstream tasks. In this pa- per, we inspect the learning dynamics of BERT fine-tuning with two indicators. We use JS divergence to detect the change of the atten- tion mode and use SVCCA distance to exam- ine the change to the feature extraction mode during BERT fine-tuning. We conclude that BERT fine-tuning mainly changes the atten- tion mode of the last layers and modifies the feature extraction mode of the intermediate and last layers. Moreover, we analyze the con- sistency of BERT fine-tuning between differ- ent random seeds and different datasets. In summary, we provide a distinctive understand- ing of the learning dynamics of BERT fine- tuning, which sheds some light on improving the fine-tuning results. 1 Introduction BERT (Bidirectional Encoder Representations from Transformers; Devlin et al. 2019) is a large pre-trained language model. It obtains state-of-the- art results on a wide array of Natural Language Processing (NLP) tasks. Unlike other previous pre- trained language models (Peters et al., 2018a; Rad- ford et al., 2018), BERT employs the multi-layer bidirectional Transformer encoder as the model architecture and proposes two novel pre-training tasks: the masked language modeling and the next sentence prediction. There are two approaches to adapt the pre- trained language representations to the down- stream tasks. One is the feature-based approach, where the parameters of the original pre-trained ∗Contribution during internship at Microsoft Research. model are frozen when applied on the downstream tasks (Mikolov et al., 2013; Pennington et al., 2014; Peters et al., 2018a). Another one is the fine-tuning approach, where the pre-trained model and the task- specific model are trained together (Dai and Le, 2015; Howard and Ruder, 2018; Radford et al., 2018). Take the classification task as an example, the new parameter added for BERT fine-tuning is a task-specific fully-connected layer, then all pa- rameters of BERT and the classification layer are trained together to minimize the loss function. Peters et al. (2019) demonstrate that the fine- tuning approach of BERT generally outperforms the feature-based approach. We know that BERT encodes task-specific representations during fine- tuning, but it is unclear about the learning dynamics of BERT fine-tuning, i.e., how fine-tuning helps BERT to improve performance on downstream tasks. We investigate the learning dynamics of BERT fine-tuning with two indicators. First, we use Jensen-Shannon divergence to measure the change of the attention mode during BERT fine-tuning. Second, we use Singular Vector Canonical Cor- relation Analysis (SVCCA; Raghu et al. (2017)) distance to measure the change …

Content truncated to 500 words