Continual Prompt Tuning for Dialog State Tracking

Continual Prompt Tuning for Dialog State Tracking Qi Zhu1, Bing Li1, Fei Mi2, Xiaoyan Zhu1, Minlie Huang1∗ 1CoAI Group, DCST, IAI, BNRIST, Tsinghua University 2Huawei Noah’s Ark Lab zhu-q18@mails.tsinghua.edu.cn, aihuang@tsinghua.edu.cn Abstract A desirable dialog system should be able to continually learn new skills without forgetting old ones, and thereby adapt to new domains or tasks in its life cycle. However, continually training a model often leads to a well-known catastrophic forgetting issue. In this paper, we present Continual Prompt Tuning, a parameter- efficient framework that not only avoids for- getting but also enables knowledge transfer be- tween tasks. To avoid forgetting, we only learn and store a few prompt tokens’ embeddings for each task while freezing the backbone pre-trained model. To achieve bi-directional knowledge transfer among tasks, we propose several techniques (continual prompt initial- ization, query fusion, and memory replay) to transfer knowledge from preceding tasks and a memory-guided technique to transfer knowl- edge from subsequent tasks. Extensive exper- iments demonstrate the effectiveness and effi- ciency of our proposed method on continual learning for dialog state tracking, compared with state-of-the-art baselines. 1 Introduction Recently, most studies have focused on developing dialog systems for specific domains in an offline manner, assuming the data distribution stays the same. However, this is far from realistic because a deployed dialog system is often required to support new domains and provide more services constantly over time. Therefore, it is crucial for a dialog sys- tem to continually learn new tasks without forget- ting old ones with high efficiency. Previous studies on continual learning (Kirk- patrick et al., 2017; Li and Hoiem, 2018) mainly focused on solving the catastrophic forgetting (CF) problem (McCloskey and Cohen, 1989): when a neural model is trained on a sequence of tasks, new tasks may interfere catastrophically with old tasks. Simply storing a model version for each task to *Corresponding author. USER: I'd like to find a place to eat. SYSTEM: In which city are you looking for the restaurant and do you have any preferred cuisine? USER: Find me Ethiopian cuisine in Berkeley. Dialog: restaurants Service name: City in which the restaurant is located: <X>. The amount of money to transfer: <Y>. Cuisine of food served in the restaurant: <Z>. … Query: (b) Continual Prompt Tuning !%" Backward transfer Prompt Input Pre-trained Model Task 1 Task 2 Task 3 transform forward transfer backward transfer Prompt Input Pre-trained Model Task 1 Task 2 Task 3 transform forward transfer backward transfer 26C2A1 Figure 1: An illustration of Continual Prompt Tuning. We train a soft prompt for each task and freeze the pre-trained model. Several techniques are proposed to transfer knowledge from preceding tasks (green solid arrows) and subsequent tasks (red dashed arrows). mitigate forgetting is prohibitive as the number of tasks grows, especially when the model size is large. To mitigate catastrophic forgetting with low computation and storage overhead, recent methods freeze the backbone model and propose to train a weight/feature mask (Mallya et al., 2018; Geng et al., 2021) or an …

Content truncated to 500 words