Intent Detection and Discovery from User Logs via Deep Semi-Supervised Contrastive Clustering
21292Authors
nan
Abstract
AbstractIntent Detection is a crucial component of Dialogue Systems wherein the objective is to classify a user utterance into one of multiple pre-defined intents. A pre-requisite for developing an effective intent identifier is a training dataset labeled with all possible user intents. However, even skilled domain experts are often unable to foresee all possible user intents at design time and for practical applications, novel intents may have to be inferred incrementally on-the-fly from user utterances. Therefore, for any real-world dialogue system, the number of intents increases over time and new intents have to be discovered by analyzing the utterances outside the existing set of intents. In this paper, our objective is to i) detect known intent utterances from a large number of unlabeled utterance samples given a few labeled samples and ii) discover new unknown intents from the remaining unlabeled samples. Existing SOTA approaches address this problem via alternate representation learning and clustering wherein pseudo labels are used for updating the representations and clustering is used for generating the pseudo labels. Unlike existing approaches that rely on epoch wise cluster alignment, we propose an end-to-end deep contrastive clustering algorithm that jointly updates model parameters and cluster centers via supervised and self-supervised learning and optimally utilizes both labeled and unlabeled data. Our proposed approach outperforms competitive baselines on five public datasets for both settings: (i) where the number of undiscovered intents are known in advance, and (ii) where the number of intents are estimated by an algorithm. We also propose a human-in-the-loop variant of our approach for practical deployment which does not require an estimate of new intents and outperforms the end-to-end approach.
Keywords
Full Content
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1836 - 1853 July 10-15, 2022 ©2022 Association for Computational Linguistics Intent Detection and Discovery from User Logs via Deep Semi-Supervised Contrastive Clustering Rajat Kumar, Mayur Patidar, Vaibhav Varshney, Lovekesh Vig, Gautam Shroff TCS Research, New Delhi, India {k.rajat2, patidar.mayur, varshney.v, lovekesh.vig, gautam.shroff}@tcs.com Abstract Intent Detection is a crucial component of Dia- logue Systems wherein the objective is to clas- sify a user utterance into one of the multiple pre-defined intents. A pre-requisite for devel- oping an effective intent identifier is a training dataset labeled with all possible user intents. However, even skilled domain experts are often unable to foresee all possible user intents at de- sign time and for practical applications, novel intents may have to be inferred incrementally on-the-fly from user utterances. Therefore, for any real-world dialogue system, the number of intents increases over time and new intents have to be discovered by analyzing the utter- ances outside the existing set of intents. In this paper, our objective is to i) detect known intent utterances from a large number of unlabeled utterance samples given a few labeled samples and ii) discover new unknown intents from the remaining unlabeled samples. Existing SOTA approaches address this problem via alternate representation learning and clustering wherein pseudo labels are used for updating the repre- sentations and clustering is used for generating the pseudo labels. Unlike existing approaches that rely on epoch-wise cluster alignment, we propose an end-to-end deep contrastive clus- tering algorithm that jointly updates model pa- rameters and cluster centers via supervised and self-supervised learning and optimally utilizes both labeled and unlabeled data. Our proposed approach outperforms competitive baselines on five public datasets for both settings: (i) where the number of undiscovered intents is known in advance, and (ii) where the number of intents is estimated by an algorithm. We also propose a human-in-the-loop variant of our approach for practical deployment which does not require an estimate of new intents and outperforms the end-to-end approach. 1 Introduction Modern dialogue systems (Louvan and Magnini, 2020) are increasingly reliant on intent detection I1: Activate My Card I2: Card Linking User-1 U11: How do I link a new card in the app |I2|0.90|Pos U12: The ATM didnt return my card! | I2|0.10|Pos U13: The ATM machine ate my card. |I1|0.25|None U14: What do I do if my card is stolen? | I1|0.9|Neg U15: I need to report a stolen card | I2|0.20|Neg U16: How is the weather today? | I2|0.10|None User-2 U21: Where do I go to add a new card?|I1|0.30|None U22: I want to activate my new card. |I1|0.95|None U23: The ATM sucked my card in. | I2| 0.20|Neg U24: My card has gone missing. | I1| 0.15 |None U25: My card was stolen | I2| 0.85|None U12, U13, U14, U15, U16, U21, U23, U24 User utterances for Human Review U12, U13, U14, U15 U21, U23, U24, U16 I3: Card Lost or …
Content truncated to 500 words
Labeled Text
AbstractIntent Detection is a crucial component of Dialogue Systems wherein the objective is to classify a user utterance into one of multiple pre-defined intents. A pre-requisite for developing an effective intent identifier is a training dataset labeled with all possible user intents. However, even skilled domain experts are often unable to foresee all possible user intents at design time and for practical applications, novel intents may have to be inferred incrementally on-the-fly from user utterances. Therefore, for any real-world dialogue system, the number of intents increases over time and new intents have to be discovered by analyzing the utterances outside the existing set of intents. In this paper, our objective is to i) detect known intent utterances from a large number of unlabeled utterance samples given a few labeled samples and ii) discover new unknown intents from the remaining unlabeled samples. Existing SOTA approaches address this problem via alternate representation learning and clustering wherein pseudo labels are used for updating the representations and clustering is used for generating the pseudo labels. Unlike existing approaches that rely on epoch wise cluster alignment, we propose an end-to-end deep contrastive clustering algorithm that jointly updates model parameters and cluster centers via supervised and self-supervised learning and optimally utilizes both labeled and unlabeled data. Our proposed approach outperforms competitive baselines on five public datasets for both settings: (i) where the number of undiscovered intents are known in advance, and (ii) where the number of intents are estimated by an algorithm. We also propose a human-in-the-loop variant of our approach for practical deployment which does not require an estimate of new intents and outperforms the end-to-end approach.
Publication Details
- Paper ID:
- 21292
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan