Transformer Growth
method
ID: transformergrowth
Related Papers
On the Transformer Growth for Progressive BERT Training
AbstractAs the excessive pre-training cost arouses the need to improve efficiency, considerable efforts have been made to train BERT progressively–start from an inferior but low-cost model and gradually increase the …...