Local Attention
method
ID: localattention
Related Papers
Improving Slot Filling Performance with Attentive Neural Networks on Dependency Structures
AbstractSlot Filling (SF) aims to extract the values of certain types of attributes (or slots, such as person:cities_of_residence) for a given entity from a large collection of source documents. In …...
Learning Adaptive Axis Attentions in Fine-tuning: Beyond Fixed Sparse Attention Patterns
AbstractWe present a comprehensive study of sparse attention patterns in Transformer models. We first question the need for pre-training with sparse attention and present experiments showing that an efficient fine-tuning …...