PMI маскировка
method
ID: pmimasking
Related Papers
Should You Mask 15% in Masked Language Modeling?
AbstractMasked language models (MLMs) conventionally mask 15% of tokens due to the belief that more masking would leave insufficient context to learn good representations; this masking rate has been widely …...