Language Understanding Tasks
Related Papers
English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too
AbstractIntermediate-task training—fine-tuning a pretrained model on an intermediate task before fine-tuning again on the target task—often improves model performance substantially on language understanding tasks in monolingual English settings. We investigate …...
Unsupervised Fine-tuning for Text Clustering
AbstractFine-tuning with pre-trained language models (e.g. BERT) has achieved great success in many language understanding tasks in supervised settings (e.g. text classification). However, relatively little work has been focused on …...
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning
AbstractAlthough pretrained language models can be fine-tuned to produce state-of-the-art results for a very wide range of language understanding tasks, the dynamics of this process are not well understood, especially …...
Word Order Does Matter and Shuffled Language Models Know It
AbstractRecent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the importance of word order information. Somewhat counter-intuitively, …...