Deep Unknown Intent Detection with Margin Loss

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5491–5496 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5491 Deep Unknown Intent Detection with Margin Loss Ting-En Lin, Hua Xu State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology lte17@mails.tsinghua.edu.cn, xuhua@tsinghua.edu.cn Abstract Identifying the unknown (novel) user intents that have never appeared in the training set is a challenging task in the dialogue system. In this paper, we present a two-stage method for detecting unknown intents. We use bidirec- tional long short-term memory (BiLSTM) net- work with the margin loss as the feature ex- tractor. With margin loss, we can learn dis- criminative deep features by forcing the net- work to maximize inter-class variance and to minimize intra-class variance. Then, we feed the feature vectors to the density-based novelty detection algorithm, local outlier factor (LOF), to detect unknown intents. Experiments on two benchmark datasets show that our method can yield consistent improvements compared with the baseline methods. 1 Introduction In the dialogue system, it is essential to identify the unknown intents that have never appeared in the training set. We can use those unknown intents to discover potential business opportunities. Be- sides, it can provide guidance for developers and accelerate the system development process. How- ever, it is also a challenging task. On the one hand, it is often difficult to obtain prior knowledge about unknown intents due to lack of examples. On the other hand, it is hard to estimate the exact num- ber of unknown intents. In addition, since user in- tents are strongly guided by prior knowledge and context, modeling high-level semantic concepts of intent is still problematic. Few previous studies are related to unknown in- tents detection. For example, Kim and Kim (2018) try to optimize the intent classifier and out-of- domain detector jointly, but out-of-domain sam- ples are still needed. The generative method (Yu et al., 2017) try to generate positive and negative examples from known classes by using adversar- ial learning to augment training data. However, the method does not work well in the discrete data space like text, and a recent study (Nalisnick et al., 2019) suggests that this approach may not work well on real-world data. Brychcin and Kr´al try to model intents through clustering. Still, it does not make good use of prior knowledge provided by known intents, and clustering results are usually unsatisfactory. Although there is a lack of prior knowledge about unknown intents, we can still leverage the advantage of known label information. Scheirer et al. (2013); Fei and Liu (2016) suggest that a m-class classifier should be able to reject exam- ples from unknown class while performing m- class classification tasks. The reason is that not all test classes have appeared in the training set, which forms a (m+1)-class classification problem where the (m+1)th class represents the unknown class. This task …

Content truncated to 500 words