Decoder Tuning: Efficient Language Understanding as Decoding

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 15072–15087 July 9-14, 2023 ©2023 Association for Computational Linguistics Decoder Tuning: Efficient Language Understanding as Decoding Ganqu Cui1, Wentao Li1, Ning Ding1, Longtao Huang2, Zhiyuan Liu1,3∗, Maosong Sun1,3∗ 1 NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing 2 Alibaba Group 3 IICTUS, Shanghai cgq22@mails.tsinghua.edu.cn Abstract With the evergrowing sizes of pre-trained mod- els (PTMs), it has been an emerging practice to only provide the inference APIs for users, namely model-as-a-service (MaaS) setting. To adapt PTMs with model parameters frozen, most current approaches focus on the input side, seeking for powerful prompts to stimulate mod- els for correct answers. However, we argue that input-side adaptation could be arduous due to the lack of gradient signals and they usu- ally require thousands of API queries, resulting in high computation and time costs. In light of this, we present Decoder Tuning (DecT), which in contrast optimizes task-specific de- coder networks on the output side. Specif- ically, DecT first extracts prompt-stimulated output scores for initial predictions. On top of that, we train an additional decoder net- work on the output representations to incor- porate posterior data knowledge. By gradient- based optimization, DecT can be trained within several seconds and requires only one PTM query per sample. Empirically, we conduct extensive natural language understanding ex- periments and show that DecT significantly outperforms state-of-the-art algorithms with a 200× speed-up. Our codes are available at https://github.com/thunlp/DecT. 1 Introduction Recent advances in pre-trained models (PTMs) demonstrate the power of the “pre-training- fine-tuning” paradigm, which empowers broad downstream NLP tasks with a single backbone model (Devlin et al., 2019; Raffel et al., 2020; Rad- ford et al., 2019). Given the million even billion- scale models, model-as-a-service (MaaS) has be- come an emerging practice in deploying massive PTMs, where users can only get access to model inference APIs (Brown et al., 2020; Sun et al., 2022b). Under such a scenario, PTMs’ parameters ∗Corresponding Author. Figure 1: Accuracy v.s. training time for MaaS adapta- tion methods under different training shots. We plot {1, 4, 16, 64, 256}-shot DecT, 0-shot Prompt, 1-shot ICL and {1, 4, 16}-shot for other algorithms. DecT outperforms all baselines by a large margin with a 200×speed-up. are frozen, and users cannot fine-tune the model on downstream tasks for adaptation. To find an alter- native way, researchers have studied MaaS PTM adaptation methods extensively. Most existing approaches in this line are based on prompts, which modify inputs with specific pat- terns. By wrapping inputs into cloze-style ques- tions or prepending inputs with a few demonstra- tive examples, PTMs could produce the right out- puts directly and show strong “in-context” learning abilities (Petroni et al., 2019; Brown et al., 2020) without any parameter update. Besides heuristic prompt design, some recent works try to optimize the input prompts without gradients. Among them, Black-box Tuning (BBT) (Sun et al., 2022b) and BBTv2 (Sun et al., 2022a) apply evolutionary al- gorithm (Hansen and Ostermeier, 2001) …

Content truncated to 500 words