Mixture of Attention Heads

Back to List
method ID: mixtureofattentionheads

Entity Details

Basic Information

Entity ID

mixtureofattentionheads

Entity Type

Method

English Label

Mixture of Attention Heads

Russian Label

Смесь голов внимания

Kazakh Label

Назар аудару бастарының қоспасы

Frequency

1

Description

English Description

An architecture that combines multi-head attention with the Mixture of Experts mechanism.

Russian Description

-

Kazakh Description

-