Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models

8451

Knowledge Triples

Source Entity	Relationship	Target Entity
Composite Attention Method	appliedIn	BERT Method
Composite Attention Method	improves	Natural Language Tasks Task
Depthwise Separable Convolutions Method	usedIn	Language Model Pre-Training Method
Dynamic Lightweight Convolutions Method	usedIn	Language Model Pre-Training Method