Masking Rate
metric
ID: maskingrate
Related Papers
Should You Mask 15% in Masked Language Modeling?
AbstractMasked language models (MLMs) conventionally mask 15% of tokens due to the belief that more masking would leave insufficient context to learn good representations; this masking rate has been widely …...