LoRA

AbstractStandard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of the PLM weights for …...

AbstractMost downstream adaptation methods tune all or part of the parameters of pre-trained models (PTMs) through gradient descent, where the tuning cost increases linearly with the growth of the model …...

AbstractFine-tuning pre-trained language models for multiple tasks can be expensive in terms of storage. Parameter-efficient transfer learning (PETL) methods have been proposed to address this issue, but they still require …...

Named entity recognition (NER) is a sub- task in the field of information extraction in natural language processing (NLP). Its main goal is to recognize named entities in text and …...