Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm

39360

Knowledge Triples

Source Entity	Relationship	Target Entity
GLUEBenchmark Material	usedIn	Ablation Studies Method
Progressive Knowledge Distillation Method	improves	Pruning Method
Pruning Method	appliesTo	Transformer Based Language Models Method