Дополнительное предварительное обучение
method
ID: furtherpretraining
Related Papers
Different Strokes for Different Folks: Investigating Appropriate Further Pre-training Approaches for Diverse Dialogue Tasks
AbstractLoading models pre-trained on the large-scale corpus in the general domain and fine-tuning them on specific downstream tasks is gradually becoming a paradigm in Natural Language Processing. Previous investigations prove …...
Aggretriever: A Simple Approach to Aggregate Textual Representations for Robust Dense Passage Retrieval
AbstractPre-trained language models have been successful in many knowledge-intensive NLP tasks. However, recent work has shown that models such as BERT are not “structurally ready” to aggregate textual information into …...
DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domains
AbstractIn recent years, pre-trained language models (PLMs) achieve the best performance on a wide range of natural language processing (NLP) tasks. While the first models were trained on general domain …...