Roles and Utilization of Attention Heads in Transformer-based Neural Language Models

37830

Knowledge Triples

Source Entity	Relationship	Target Entity
Downstream Tasks Task	uses	BERT Method
Downstream Tasks Task	uses	ELECTRA Method
Downstream Tasks Task	uses	GPT Method
Downstream Tasks Task	uses	GPT-2 Material
Sentence Encoders Method	appliesTo	Downstream Tasks Task
Sentence Encoders Method	examines	ProbingTasks Task
Sentence Encoders Method	isBasedOn	Transformer Architecture Method
Transformer Architecture Method	uses	Multi-Head Attention Method