Эффективная парадигма KD
method
ID: efficientkdparadigm
Related Papers
Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation
AbstractRecently, knowledge distillation (KD) has shown great success in BERT compression. Instead of only learning from the teacher’s soft label as in conventional KD, researchers find that the rich information …...