Hierarchical Relational Knowledge Distillation

AbstractOn many natural language processing tasks, large pre-trained language models (PLMs) have shown overwhelming performances compared with traditional neural network methods. Nevertheless, their huge model size and low inference speed …...