Towards Fine-tuning Pre-trained Language Models with Integer Forward and Backward Propagation

Findings of the Association for Computational Linguistics: EACL 2023, pages 1912–1921 May 2-6, 2023 ©2023 Association for Computational Linguistics Towards Fine-tuning Pre-trained Language Models with Integer Forward and Backward Propagation Mohammadreza Tayaranian∗1 Alireza Ghaffari∗1 Marzieh S. Tahaei1 Mehdi Rezagholizadeh1 Masoud Asgharian2 Vahid Partovi Nia1 1 Huawei Noah’s Ark Lab, Montreal Research Center 2 Department of Mathematics and Statistics, McGill University {mohammadreza.tayaranian, alireza.ghaffari, marzieh.tahaei}@huawei.com {mehdi.rezagholizadeh, vahid.partovinia}@huawei.com masoud.asgharian2@mcgill.ca Abstract The large number of parameters of some promi- nent language models, such as BERT, makes their fine-tuning on downstream tasks compu- tationally intensive and energy hungry. Previ- ously researchers were focused on lower bit- width integer data types for the forward propa- gation of language models to save memory and computation. As for the backward propagation, however, only 16-bit floating-point data type has been used for the fine-tuning of BERT. In this work, we use integer arithmetic for both forward and back propagation in the fine-tuning of BERT. We study the effects of varying the integer bit-width on the model’s metric perfor- mance. Our integer fine-tuning uses integer arithmetic to perform forward propagation and gradient computation of linear, layer-norm, and embedding layers of BERT. We fine-tune BERT using our integer training method on SQuAD v1.1 and SQuAD v2., and GLUE benchmark. We demonstrate that metric performance of fine-tuning 16-bit integer BERT matches both 16-bit and 32-bit floating-point baselines. Fur- thermore, using the faster and more memory efficient 8-bit integer data type, integer fine- tuning of BERT loses an average of 3.1 points compared to the FP32 baseline. 1 Introduction Over the past few years, integration of attention mechanisms into deep learning models led to the creation of transformer based models. BERT (De- vlin et al., 2018) is a prominent transformer based language model which has shown state-of-the-art performance in natural language processing (NLP) tasks. BERT requires high memory and computational resources due to its large number of parameters. Having large number of parameters incurs chal- lenges for inference, training, and also fine-tuning *Equal contribution. FP32 Int32 Int16 Int8 0 50 100 150 Joules CPU Energy Consumption for 1 Billion Operations Addition Multiplication FP32 Int32 Int16 Int8 0.0 0.5 1.0 1.5 Seconds CPU Latency for 1 Billion Operations Addition Multiplication Figure 1: Energy consumption and latency of 1 billion operations using various data types, measured on an Intel® Xeon® CPU E5-2698 v4. of this model. Moreover, the training phase i.e. pre- training and fine-tuning, involves more operations compared to the inference. More specifically, the training phase includes gradient computation and weight update that make the training more compu- tationally intensive. One method of reducing the computational com- plexity of deep learning models is to represent their parameters and activations in low bit-width data types. This reduces the memory footprint of the model and enables more efficient computations. For instance, Figure 1 shows that low-bit integer data types have higher throughput and better energy consumption compared to floating-point. Previous research attempts at integer quantiza- tion of transformer based language models were only focused on forward …

Content truncated to 500 words