Attention-Based Layer Projection

Back to List
method ID: attentionbasedlayerprojection

Related Papers

Universal-KD: Attention-based Output-Grounded Intermediate Layer Knowledge Distillation

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractIntermediate layer matching is shown as an effective approach for improving knowledge distillation (KD). However, this technique applies matching in the hidden spaces of two different networks (i.e. student and …...

31331