Multi-Head Attention
method
ID: multiheadattention
Outgoing Relationships
| Relationship | Target Entity | Type | Actions |
|---|---|---|---|
| composedOf | Attention Head | otherentity | View |
| inspiredBy | Grouped Head Attention | method | View |
| componentOf | Transformer | method | View |
| leadsTo | Informative Representations | metric | View |
| suffersFrom | Attention Collapse | otherentity | View |
| affects | AlignmentQuality | metric | View |
| extends | Neural Attention Mechanism | method | View |
| relatedTo | Mixture of Attentive Experts | method | View |
| partOf | Transformers | method | View |
| addresses | Long Distance Dependencies | otherentity | View |