Open-Topic False Information Detection on Social Networks with Contrastive Adversarial Learning

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 261 - 276 December 7-11, 2022 ©2022 Association for Computational Linguistics Estimating Soft Labels for Out-of-Domain Intent Detection Hao Lang ∗ Yinhe Zheng ∗ † Jian Sun Fei Huang Luo Si Yongbin Li † Alibaba Group {hao.lang, jian.sun, f.huang, luo.si,shuide.lyb}@alibaba-inc.com, zhengyinhe1@163.com Abstract Out-of-Domain (OOD) intent detection is im- portant for practical dialog systems. To allevi- ate the issue of lacking OOD training samples, some works propose synthesizing pseudo OOD samples and directly assigning one-hot OOD labels to these pseudo samples. However, these one-hot labels introduce noises to the train- ing process because some “hard” pseudo OOD samples may coincide with In-Domain (IND) intents. In this paper, we propose an adaptive soft pseudo labeling (ASoul) method that can estimate soft labels for pseudo OOD samples when training OOD detectors. Semantic con- nections between pseudo OOD samples and IND intents are captured using an embedding graph. A co-training framework is further intro- duced to produce resulting soft labels following the smoothness assumption, i.e., close samples are likely to have similar labels. Extensive ex- periments on three benchmark datasets show that ASoul consistently improves the OOD de- tection performance and outperforms various competitive baselines. 1 Introduction Intent detection is essential for dialogue systems, and current methods usually achieve high perfor- mance under the closed-world assumption (Shu et al., 2017), i.e., data distributions are static, and only a fixed set of intents are considered. However, such an assumption may not be valid in practice, where we usually confront an open-world (Fei and Liu, 2016), i.e., unknown intents that are not trained may emerge. It is necessary to equip dialogue sys- tems with Out-of-Domain (OOD) detection abil- ities so that they can accurately classify known In-Domain (IND) intents while rejecting unknown OOD intents (Yan et al., 2020a; Shen et al., 2021). A major challenge for OOD detection is the lack of OOD samples (Xu et al., 2020a). In most appli- ∗ Equal contribution. † Corresponding author. I wish to know my exact money How much money do I earn Tell me my exact salary One‐hot OOD Label 0.0 0.0 0.0 0.0 1.0 ASoul 0.15 0.21 0.10.26 0.28 × √ I wish to know your hobbies Pseudo OOD Sample IND Sample IND Distribution OOD class k IND class OOD class k IND class Figure 1: A pseudo OOD sample generated by distort- ing IND inputs (See more examples in Appendix A). Comparing to the one-hot OOD label, the soft label pro- duced by ASoul is more suitable for this pseudo OOD sample since it carries some IND intents. cations, it is hard, if not impossible, to collect OOD samples from the test distribution before training (Du et al., 2021). To tackle this issue, various stud- ies try to synthesize pseudo OOD samples in the training process. Existing methods include dis- torting IND samples (Choi et al., 2021; Shu et al., 2021; Ouyang et al., 2021), using generative mod- els (Ryu et al., 2018; …

Content truncated to 500 words