Continual Sequence Generation with Adaptive Compositional Modules

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3653 - 3667 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Continual Sequence Generation with Adaptive Compositional Modules Yanzhe Zhang Georgia Institute of Technology z_yanzhe@gatech.edu Xuezhi Wang Google xuezhiw@google.com Diyi Yang Georgia Institute of Technology dyang888@gatech.edu Abstract Continual learning is essential for real-world deployment when there is a need to quickly adapt the model to new tasks without forget- ting knowledge of old tasks. Existing work on continual sequence generation either al- ways reuses existing parameters to learn new tasks, which is vulnerable to catastrophic for- getting on dissimilar tasks, or blindly adds new parameters for every new task, which could prevent knowledge sharing between sim- ilar tasks. To get the best of both worlds, in this work, we propose continual sequence generation with adaptive compositional mod- ules to adaptively add modules in transformer architectures and compose both old and new modules for new tasks. We also incorporate pseudo experience replay to facilitate knowl- edge transfer in those shared modules. Exper- iment results on various sequences of genera- tion tasks show that our framework can adap- tively add modules or reuse modules based on task similarity, outperforming state-of-the-art baselines in terms of both performance and pa- rameter efficiency. We make our code pub- lic at https://github.com/GT-SALT/ Adaptive-Compositional-Modules. 1 Introduction Current state-of-the-art language generation mod- els can achieve great performance on a wide range of sequence generation tasks (Radford et al., 2019; Lewis et al., 2020) with a static data distribution. However, real-world scenarios are often changing which requires the model to learn with dynamic data distributions. In such cases of data distribu- tions shift, current generation models often suf- fer from catastrophic forgetting (Sun et al., 2019): models completely and abruptly forget previously learned information upon learning new information. Continual learning (CL) (Ring, 1998; Thrun, 1998) has been introduced to improve model’s ability to learn tasks in a stream by mitigating forgetting Figure 1: Comparison between previous methods (a and b) and our proposed method (c), from a multi-layer transformer model perspective. The blue blocks re- fer to learnable modules and the yellow blocks refer to frozen pretrained modules . a: retrain the whole model every time when new tasks arrive. b: insert task- specific modules for each task, while keeping the pre- trained model frozen. c: detect reusable old modules and add new modules adaptively. and facilitating knowledge transfer (Lopez-Paz and Ranzato, 2017), however, continual sequence gen- eration is relatively under-investigated. Comparing to continual learning on text classifi- cation and question answering (Wang et al., 2020; Holla et al., 2020; Huang et al., 2021), continual sequence generation is more challenging, since the output is no longer discrete labels but sequential text data in different styles/domains. Based on how to retain old knowledge while learning new tasks, current continual sequence generation methods can be categorized into two types. The first one con- tinually learns new tasks on old parameters (Fig 1 a), with …

Content truncated to 500 words