Adversarial Self-Supervised Learning for Out-of-Domain Detection

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5631–5639 June 6–11, 2021. ©2021 Association for Computational Linguistics 5631 Adversarial Self-Supervised Learning for Out-of-Domain Detection Zhiyuan Zeng1, Keqing He1,2, Yuanmeng Yan1, Hong Xu1, Weiran Xu1∗ 1Pattern Recognition & Intelligent System Laboratory 1Beijing University of Posts and Telecommunications, Beijing, China 2Meituan Group, Beijing, China {zengzhiyuan,yanyuanmeng,xuhong,xuweiran}@bupt.edu.cn {hekeqing}@meituan.com Abstract Detecting out-of-domain (OOD) intents is cru- cial for the deployed task-oriented dialogue system. Previous unsupervised OOD detec- tion methods only extract discriminative fea- tures of different in-domain intents while su- pervised counterparts can directly distinguish OOD and in-domain intents but require ex- tensive labeled OOD data. To combine the benefits of both types, we propose a self- supervised contrastive learning framework to model discriminative semantic features of both in-domain intents and OOD intents from unla- beled data. Besides, we introduce an adversar- ial augmentation neural module to improve the efficiency and robustness of contrastive learn- ing. Experiments on two public benchmark datasets show that our method can consistently outperform the baselines with a statistically significant margin.1 1 Introduction Task-oriented dialog systems (Sarikaya, 2017; Akasaki and Kaji, 2017; Gnewuch et al., 2017; Shum et al., 2018; Tulshan and Dhage, 2018) such as Google’s DialogFlow or Amazon’s Lex have become ubiquitous to make people interact with machines using natural language. In the architec- ture of a dialogue system, detecting unknown or OOD (Out-of-Domain) intents from user queries is an essential component that aims to know when a user query falls outside their range of predefined supported intents. Different from traditional intent detection tasks, we do not know the exact num- ber of unknown intents in practical scenarios and can barely annotate extensive OOD samples. Lack of real OOD examples always leads to poor prior knowledge about these unknown intents, making it ∗Weiran Xu is the corresponding author. 1Our code is available at https://github.com/p arZival27/Adversarial-Self-Supervised-Ou t-of-Domain-Detection. challenging to identify OOD samples in the task- oriented dialog system. Previous methods of detecting OOD intents can be generally classified into two types: unsuper- vised and supervised OOD detection. Unsuper- vised OOD detection (Breunig et al., 2000; Bendale and Boult, 2016; Hendrycks and Gimpel, 2017; Shu et al., 2017; Lee et al., 2018; Ren et al., 2019a; Lin and Xu, 2019; Snell et al., 2017; Finn et al., 2017; Xu et al., 2020) means no labeled OOD samples except for labeled in-domain data. By contrast, su- pervised OOD detection (Scheirer et al., 2013; Fei and Liu, 2016; Kim and Kim, 2018; Larson et al., 2019; He et al., 2020b; Zheng et al., 2020) repre- sents that there are extensive labeled OOD samples in the training data. Most of unsupervised OOD detection methods follow a two-stage framework: training and de- tecting. They first train an in-domain intent clas- sifier to extract intent representations, then detect whether the test query belongs to OOD by estimat- ing its probability density. For example, Hendrycks and Gimpel (2017); Shu et al. (2017) simply use …

Content truncated to 500 words