Матрицалық көбейту операторы
method
ID: matrixproductoperator
Related Papers
Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators
AbstractThis paper presents a novel pre-trained language models (PLM) compression approach based on the matrix product operator (short as MPO) from quantum many-body physics. It can decompose an original matrix …...
Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models
AbstractRecently, Mixture-of-Experts (short as MoE) architecture has achieved remarkable success in increasing the model capacity of large-scale language models. However, MoE requires incorporating significantly more parameters than the base model …...
Small Pre-trained Language Models Can be Fine-tuned as Large Models via Over-Parameterization
AbstractBy scaling the model size, large pre-trained language models (PLMs) have shown remarkable performance in various natural language processing tasks, mostly outperforming small PLMs by a large margin. However, due …...