Foreign Language Tokens

AbstractEnglish pretrained language models, which make up the backbone of many modern NLP systems, require huge amounts of unlabeled training data. These models are generally presented as being trained only …...