OPT models

AbstractScaling up language models has led to unprecedented performance gains, but little is understood about how the training dynamics change as models get larger. How do language models of different …...