Continual Learning for Natural Language Generations with Transformer Calibration

Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 40 - 49 December 7-8, 2022 ©2022 Association for Computational Linguistics Continual Learning for Natural Language Generations with Transformer Calibration Pang Yang, Dingcheng Li, Ping Li Cognitive Computing Lab Baidu Research 10900 NE 8th St. Bellevue, WA 98004, USA {pengyang5612, dingchengl, pingli98}@gmail.com Abstract Conventional natural language process (NLP) generation models are trained offline with a given dataset for a particular task, which is referred to as isolated learning. Research on sequence-to-sequence language generation aims to study continual learning model to con- stantly learning from sequentially encountered tasks. However, continual learning studies often suffer from catastrophic forgetting, a persistent challenge for lifelong learning. In this paper, we present a novel NLP transformer model that attempts to mitigate catastrophic forgetting in online continual learning from a new perspec- tive, i.e., attention calibration. We model the at- tention in the transformer as a calibrated unit in a general formulation, where the attention cal- ibration could give benefits to balance the sta- bility and plasticity of continual learning algo- rithms through influencing both their forward inference path and backward optimization path. Our empirical experiments, paraphrase gener- ation and dialog response generation, demon- strate that this work outperforms state-of-the- art models by a considerable margin and effec- tively mitigate the forgetting. 1 Introduction Sequence-to-sequence (Seq2Seq) generation has been widely applied in artificial learning (AI) sys- tem to deal with various challenging tasks, e.g., paraphrase, dialogue system (Bordes et al., 2016), machine translation, etc. In addition, powerful rep- resentation learning (e.g., Transformer) have been used in Seq2Seq models, which have taken the state-of-the-art of generation models to a new level. Generally, nature language generation (NLG) mod- els leverage an encoder to create a vector repre- sentation for source inputs, and then pass this rep- resentation into a decoder so as to output a target sequence word by word. For example, Bart (Lewis et al., 2019) is such a transformer-based NLG ar- chitecture that is equipped with the BERT-type net- work structure (Devlin et al., 2019) as its encoder and with the GPT-type structure as the decoder. Despite the remarkable ability on sequence gen- eration, the conventional paradigm aims to learn a Seq2Seq model on the whole available dataset, which limits its ability in accumulating knowledge in continual learning scenario. When switching to a new task from some previously learned ones, the fine-tuned model on the new task sometimes faces a significant performance drop on previous learned data, where such a phenomenon is also referred to as catastrophic forgetting (Parisi et al., 2019; Mai et al., 2021; Yin et al., 2021; Li et al., 2022a,b). In contrast, humans and animals exhibit remarkable ability to deal with new tasks by effectively adapt- ing their acquired knowledge without forgetting the previously learned skills. If one desires to build a human-like NLG model, continual learning ability is a necessary skill for achieving this goal. The existing replay-based continual learning approaches have taken into account of differ- ent perspectives …

Content truncated to 500 words