Rectified Linear Attention

Back to List
method ID: rectifiedlinearattention

Entity Details

Basic Information

Entity ID

rectifiedlinearattention

Entity Type

Method

English Label

Rectified Linear Attention

Russian Label

Исправленное линейное внимание

Kazakh Label

Түзетілген сызықты көңіл

Frequency

1

Description

English Description

A novel method for achieving sparsity in attention by replacing the softmax activation with a ReLU.

Russian Description

-

Kazakh Description

-