Incremental Intent Detection for Medical Domain with Contrast Replay Networks

Findings of the Association for Computational Linguistics: ACL 2022, pages 3549 - 3556 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Incremental Intent Detection for Medical Domain with Contrastive Replay Networks Guirong Bai1,2, Shizhu He1,2, Kang Liu1,2,3, Jun Zhao1,2 1 National Laboratory of Pattern Recognition, Institute of Automation, CAS 2 School of Artificial Intelligence, University of Chinese Academy of Sciences 3 Beijing Academy of Artificial Intelligence {guirong.bai, shizhu.he, kliu, jzhao}@nlpr.ia.ac.cn Abstract Conventional approaches to medical intent de- tection require fixed pre-defined intent cate- gories. However, due to the incessant emer- gence of new medical intents in the real world, such requirement is not practical. Consider- ing that it is computationally expensive to s- tore and re-train the whole data every time new data and intents come in, we propose to incre- mentally learn emerged intents while avoiding catastrophically forgetting old intents. We first formulate incremental learning for medical in- tent detection. Then, we employ a memory- based method to handle incremental learning. We further propose to enhance the method with contrastive replay networks, which use multilevel distillation and contrastive objec- tive to address training data imbalance and medical rare words respectively. Experiments show that the proposed method outperforms the state-of-the-art model by 5.7% and 9.1% of accuracy on two benchmarks respectively. 1 Introduction Medical intent detection aims to identify intents of medical queries and classify them into specific categories (Chen et al., 2012; Howard and Cam- bria, 2013; Guo et al., 2014; Cai et al., 2017). In medical scenarios, understanding query intent is very important for medical question answer system- s (Wu et al., 2020; Mrini et al., 2021). Typically, to perform medical intent detection, existing methods pre-define a class set of fixed medical intent cat- egories in advance, and train the whole collected dataset with the fixed class set. However, novel medical intents incessantly emerge with new data in the real world. When given a query with new intent category that is out of the pre-defined class set, these models can do nothing about it. A straightforward solution is to store and re-train the whole data every time new data and intents come in. However, it is almost infeasible with limit- ed storage budget and computation cost in practice Learn Intent Medical Fees Learn Intent Treatment Plan Learn Intent Cause Analysis Data for Treatment Plan Data for Medical Fees Data for Cause Analysis Figure 1: Incremental medical intent detection. (Wang et al., 2019). Consider the above problems, we propose to address this issue in a incremental learning way (Ring et al., 1994; Thrun, 1998), where the number of intent categories is allowed to incrementally increase and the system can inces- santly learn emerged novel intents from continually arriving data of new intents, which is illustrated as Figure 1. Naturally, one simple incremental method is to directly finetune the model by new intents data. However, this method suffers from the serious catastrophic forgetting problem (McCloskey and Cohen, 1989; French, 1999; Wang et al., 2019). After fitting emerged data of new …

Content truncated to 500 words