Natural Language Tasks
task
ID: naturallanguagetasks
Related Papers
Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models
AbstractIn this paper, we detail the relationship between convolutions and self-attention in natural language tasks. We show that relative position embeddings in self-attention layers are equivalent to recently-proposed dynamic lightweight …...
LeeBERT: Learned Early Exit for BERT with cross-level optimization
AbstractPre-trained language models like BERT are performant in a wide range of natural language tasks. However, they are resource exhaustive and computationally expensive for industrial scenarios. Thus, early exits are …...