Revisiting Intermediate Layer Distillation for Compressing Language Models: An Overfitting Perspective

33438

Knowledge Triples

Source Entity	Relationship	Target Entity
Consistency Regularized ILD Method	buildsOn	Intermediate Layer Distillation Method
Consistency Regularized ILD Method	mitigates	Overfitting Metric
KnowledgeDistillation Method	appliesTo	BERT Method
KnowledgeDistillation Method	evaluatedIn	GLUE Material