AutoTinyBERT

Back to List
method ID: autotinybert

Related Papers

AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractPre-trained language models (PLMs) have achieved great success in natural language processing. Most of PLMs follow the default setting of architecture hyper-parameters (e.g., the hidden dimension is a quarter of …...

8328