Consistency Regularized ILD
method
ID: consistencyregularizedild
Related Papers
Revisiting Intermediate Layer Distillation for Compressing Language Models: An Overfitting Perspective
AbstractKnowledge distillation (KD) is a highly promising method for mitigating the computational problems of pre-trained language models (PLMs). Among various KD approaches, Intermediate Layer Distillation (ILD) has been a de …...