Обрезка сети

AbstractTransformer-based pre-trained language models have significantly improved the performance of various natural language processing (NLP) tasks in the recent years. While effective and prevalent, these models are usually prohibitively large …...

This work studies the long-standing prob- lems of model capacity and negative interfer- ence in multilingual neural machine transla- tion (MNMT). We use network pruning tech- niques and observe that …...

AbstractTransformer-based pre-trained language models have demonstrated superior performance on various natural language processing tasks. However, it remains unclear how the skills required to handle these tasks distribute among model parameters. …...