Combining Parameter-efficient Modules for Task-level Generalisation

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 687–702 May 2-6, 2023 ©2023 Association for Computational Linguistics Combining Parameter-efficient Modules for Task-level Generalisation Edoardo M. Pontiϵ Alessandro Sordoniσ Yoshua Bengioµ,ρ Siva Reddyµ,γ ϵUniversity of Edinburgh µMila – Quebec AI Institute γMcGill University σMicrosoft Research Montréal ρUniversité de Montréal eponti@ed.ac.uk Abstract A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks. In this work, we assume that each task is associated with a subset of latent skills from an (arbitrary size) inventory. In turn, each skill corresponds to a parameter-efficient (sparse / low-rank) model adapter. By jointly learning adapters and a routing function that allocates skills to each task, the full network is instantiated as the average of the parameters of active skills. We propose several inductive biases that encourage re-usage and composi- tion of the skills, including variable-size skill allocation and a dual-speed learning rate. We evaluate our latent-skill model in two main set- tings: 1) multitask reinforcement learning for instruction following on 8 levels of the BabyAI platform; and 2) few-shot fine-tuning of lan- guage models on 160 NLP tasks of the CrossFit benchmark. We find that the modular design of our network enhances sample efficiency in rein- forcement learning and few-shot generalisation in supervised learning, compared to a series of baselines. These include models where param- eters are fully shared, task-specific, or condi- tionally generated (HyperFormer), as well as sparse mixture-of-experts (Task-MoE). 1 Introduction Task-level generalisation involves training a model on multiple tasks (in parallel or sequentially) and then performing zero-shot inference or few-shot adaptation on new tasks (Caruana, 1997; Ye et al., 2021). However, the training signals from different tasks may interfere with each other (McCloskey and Cohen, 1989) or lead to catastrophic forgetting of previous knowledge (French, 1999). Moreover, when the nature of the tasks varies between training and evaluation, models often struggle to generalise systematically (Hupkes et al., 2020). A potential solution to these challenges consists in a modular design of neural architectures. In Tasks Skills Skill Inventory Layer 1 Layer N Active skills x y Active Task Fine-Grained Skill Selection Figure 1: A diagram of our latent-skill model: 1) a row of the task–skill binary matrix is selected accord- ing to the active task; 2) the sparse (SFT) or low-rank (LoRA) adapters corresponding to active skills from a layer-specific inventory are combined; 3) the resulting combination is applied to the weights of a neural net- work, e.g. BART in our multi-task experiments. fact, these are endowed with an inductive bias to- wards updating and activating modules locally and asynchronously, thus preventing negative interfer- ence (Jacobs et al., 1991b). Additionally, by dis- entangling autonomous facets of knowledge (also known as skills) that are re-used and recombined in original ways for new tasks, modularity facilitates systematic generalisation (Alet et al., 2018; Ponti, 2021; Kingetsu et al., 2021, inter alia). Previous work on task-level generalisation fo- cused on …

Content truncated to 500 words