Magnitude Pruning
Related Papers
Successfully Applying the Stabilized Lottery Ticket Hypothesis to the Transformer Architecture
AbstractSparse models require less memory for storage and enable a faster inference by reducing the necessary number of FLOPs. This is relevant both for time-critical and on-device computations using neural …...
Layer-wise Model Pruning based on Mutual Information
AbstractInspired by mutual information (MI) based feature selection in SVMs and logistic regression, in this paper, we propose MI-based layer-wise pruning: for each layer of a multi-layer neural network, neurons …...
Combining Compressions for Multiplicative Size Scaling on Natural Language Tasks
AbstractQuantization, knowledge distillation, and magnitude pruning are among the most popular methods for neural network compression in NLP. Independently, these methods reduce model size and can accelerate inference, but their …...
Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training
AbstractRecent studies on the lottery ticket hypothesis (LTH) show that pre-trained language models (PLMs) like BERT contain matching subnetworks that have similar transfer learning performance as the original PLM. These …...