BigBird
method
ID: bigbird
Related Papers
LittleBird: Efficient Faster & Longer Transformer for Question Answering
AbstractBERT has shown a lot of sucess in a wide variety of NLP tasks. But it has a limitation dealing with long inputs due to its attention mechanism. Longformer, ETC …...