Masking
method
ID: masking
Related Papers
An Exploration of Placeholding in Neural Machine Translation
Phrase-based machine translation provides the system developer with controls that enable fine-grained control over machine translation output. One approach to provide similar control in neural machine transla- tion is placeholding …...
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
AbstractWe present an efficient method of utilizing pretrained language models, where we learn selective binary masks for pretrained weights in lieu of modifying them through finetuning. Extensive evaluations of masking …...
On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies
AbstractWe study how masking and predicting tokens in an unsupervised fashion can give rise to linguistic structures and downstream performance gains. Recent theories have suggested that pretrained language models acquire …...