Muppet: Massive Multi-task Representations with Pre-Finetuning

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5799–5811 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5799 Muppet: Massive Multi-task Representations with Pre-Finetuning Armen Aghajanyan Facebook armenag@fb.com Anchit Gupta Facebook anchit@fb.com Akshat Shrivastava Facebook akshats@fb.com Xilun Chen Facebook xilun@fb.com Luke Zettlemoyer Facebook lsz@fb.com Sonal Gupta Facebook sonalgupta@fb.com Abstract We propose pre-finetuning, an additional large- scale learning stage between language model pre-training and fine-tuning. Pre-finetuning is massively multi-task learning (around 50 datasets, over 4.8 million total labeled ex- amples), and is designed to encourage learn- ing of representations that generalize better to many different tasks. We show that pre- finetuning consistently improves performance for pretrained discriminators (e.g. RoBERTa) and generation models (e.g. BART) on a wide range of tasks (sentence prediction, common- sense reasoning, MRC, etc.), while also sig- nificantly improving sample efficiency during fine-tuning. We also show that large-scale multi-tasking is crucial; pre-finetuning can hurt performance when few tasks are used up until a critical point (usually above 15) after which performance improves linearly in the number of tasks. 1 Introduction The recent success of language model pre-training (Devlin et al., 2018; Liu et al., 2019b; Lewis et al., 2019; Raffel et al., 2019; Radford et al., 2019) is remarkable, at least in part, due to the exclu- sive use of self supervision, without any manu- ally labeled data. For many tasks, however, we already have training examples for related prob- lems, which we should be able to leverage. Recent work has shown gains from fine-tuning schemes that are multi-task (Raffel et al., 2019; Khashabi et al., 2020) and multi-stage (Liu et al., 2019a), but it can be difficult to know which intermediate tasks will best transfer (Raffel et al., 2019). In this paper, we show that multi-task supervised tuning, if done at a sufficiently large scale with many dif- ferent tasks, can be an effective second stage of task-agnostic pre-training, removing the need to pre-select the best intermediate tasks. More specifically, in addition to the standard pre-training/fine-tuning methodology of learning language tasks, we introduce a new intermediate stage, pre-finetuning. Pre-finetuning involves a massive multi-task learning step (4.8 million total training examples) performed on around 50 classi- fication, summarization, question answering, and common sense reasoning tasks. We believe we are the first to investigate multi-task learning at this scale in terms of both number and types of tasks. We show, in particular, that standard multi-tasking schemes can be unstable and often fail to learn high quality representations. However, we introduce a new training scheme which uses loss scaling and task-heterogeneous batches so that gradient steps are more evenly balanced across multiple differ- ent competing tasks, greatly improving training stability and overall performance. We call our pre- finetuned models MUPPET; Massive Multi-task RePresentation with PrE-fineTuning. Through extensive experiments, we show that in- corporating pre-finetuning to RoBERTa (Liu et al., 2019b) and BART (Lewis et al., 2019) models yields consistent improvements, including new state-of-the-art performance for RTE (Bentivogli et al., 2009) and HellaSWAG (Zellers et al., …

Content truncated to 500 words