Parameter Efficiency
metric
ID: parameterefficiency
Related Papers
Continual Sequence Generation with Adaptive Compositional Modules
AbstractContinual learning is essential for real-world deployment when there is a need to quickly adapt the model to new tasks without forgetting knowledge of old tasks. Existing work on continual …...
Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models
AbstractRecently, Mixture-of-Experts (short as MoE) architecture has achieved remarkable success in increasing the model capacity of large-scale language models. However, MoE requires incorporating significantly more parameters than the base model …...