Конвергенция жылдамдығы
Related Papers
Self-Paced Learning for Neural Machine Translation
AbstractRecent studies have proven that the training of neural machine translation (NMT) can be facilitated by mimicking the learning process of humans. Nevertheless, achievements of such kind of curriculum learning …...
Uncertainty-Aware Curriculum Learning for Neural Machine Translation
AbstractNeural machine translation (NMT) has proven to be facilitated by curriculum learning which presents examples in an easy-to-hard order at different training stages. The keys lie in the assessment of …...
Pre-training a BERT with Curriculum Learning by Increasing Block-Size of Input Text
AbstractRecently, pre-trained language representation models such as BERT and RoBERTa have achieved significant results in a wide range of natural language processing (NLP) tasks, however, it requires extremely high computational …...
Improving Stability of Fine-Tuning Pretrained Language Models via Component-Wise Gradient Norm Clipping
AbstractFine-tuning over large pretrained language models (PLMs) has established many state-of-the-art results. Despite its superior performance, such fine-tuning can be unstable, resulting in significant variance in performance and potential risks …...
Tricks for Training Sparse Translation Models
AbstractMulti-task learning with an unbalanced data distribution skews model learning towards high resource tasks, especially when model capacity is fixed and fully shared across all tasks. Sparse scaling architectures, such …...