Composed Variational Natural Language Generation for Few-shot Intents

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3379–3388 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3379 Composed Variational Natural Language Generation for Few-shot Intents Congying Xia1∗, Caiming Xiong2, Philip Yu1 and Richard Socher2 1University of Illinois at Chicago, Chicago, IL, USA 2Salesforce Research, Palo Alto, CA, US {cxia8, psyu}@uic.edu, {cxiong, rsocher}@salesforce.com Abstract In this paper, we focus on generating training examples for few-shot intents in the realistic imbalanced scenario. To build connections be- tween existing many-shot intents and few-shot intents, we consider an intent as a combina- tion of a domain and an action, and propose a composed variational natural language genera- tor (CLANG), a transformer-based conditional variational autoencoder. CLANG utilizes two latent variables to represent the utterances cor- responding to two different independent parts (domain and action) in the intent, and the la- tent variables are composed together to gen- erate natural examples. Additionally, to im- prove the generator learning, we adopt the con- trastive regularization loss that contrasts the in-class with the out-of-class utterance genera- tion given the intent. To evaluate the quality of the generated utterances, experiments are con- ducted on the generalized few-shot intent de- tection task. Empirical results show that our proposed model achieves state-of-the-art per- formances on two real-world intent detection datasets. 1 Introduction Intelligent assistants have gained great popularity in recent years since they provide a new way for people to interact with the Internet conversationally (Hoy, 2018). However, it is still challenging to an- swer people’s diverse questions effectively. Among all the challenges, identifying user intentions from their spoken language is important and essential for all the downstream tasks. Most existing works (Hu et al., 2009; Xu and Sarikaya, 2013; Chen et al., 2016; Xia et al., 2018) formulate intent detection as a classification task and achieve high performance on pre-defined in- tents with sufficient labeled examples. With this ∗Work was done when Congying was a research intern at Salesforce Research. ever-changing world, a realistic scenario is that we have imbalanced training data with existing many- shot intents and insufficient few-shot intents. Previ- ous intent detection models (Yin, 2020; Yin et al., 2019) deteriorate drastically in discriminating the few-shot intents. To alleviate this scarce annotation problem, sev- eral methods (Wei and Zou, 2019; Malandrakis et al., 2019; Yoo et al., 2019) have been proposed to augment the training data for low-resource spo- ken language understanding (SLU). Wei and Zou (2019) introduce simple data augmentation rules for language transformation like insert, delete and swap. Malandrakis et al. (2019) and Yoo et al. (2019) utilize variational autoencoders (Kingma and Welling, 2013) with simple LSTMs (Hochre- iter and Schmidhuber, 1997) that have limited model capacity to do text generation. Furthermore, these models are not specifically designed to trans- fer knowledge from existing many-shot intents to few-shot intents. In this paper, we focus on transferable natural language generation by learning how to compose utterances with many-shot intents and transferring to few-shot intents. When users interact with in- telligent assistants, their …

Content truncated to 500 words