RealFormer

Transformer is the backbone of modern NLP models. In this paper, we propose Real- Former, a simple and generic technique to create Residual Attention Layer Transformer networks that significantly outperform …...