Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation

467

Knowledge Triples

Source Entity	Relationship	Target Entity
BERT Method	enhancedBy	KnowledgeDistillation Method
Sentence Representation Approximating Oriented Distillation Method	enables	Transfer Learning Method
Sentence Representation Approximating Oriented Distillation Method	evaluatedIn	GLUEBenchmark Material
Sentence Representation Approximating Oriented Distillation Method	outperforms	ELMo Method
Transfer Learning Method	involves	Fine Tuning Method