Simple is Better! Lightweight Data Augmentation for Low Resource Slot Filling and Intent Classification

Simple is Better! Lightweight Data Augmentation for Low Resource Slot Filling and Intent Classification Samuel Louvan University of Trento Fondazione Bruno Kessler slouvan@fbk.eu Bernardo Magnini Fondazione Bruno Kessler magnini@fbk.eu Abstract Neural-based models have achieved outstand- ing performance on slot filling and intent clas- sification, when fairly large in-domain train- ing data are available. However, as new do- mains are frequently added, creating sizeable data is expensive. We show that lightweight augmentation, a set of augmentation meth- ods involving word span and sentence level operations, alleviates data scarcity problems. Our experiments on limited data settings show that lightweight augmentation yields signifi- cant performance improvement on slot filling on the ATIS and SNIPS datasets, and achieves competitive performance with respect to more complex, state-of-the-art, augmentation ap- proaches. Furthermore, lightweight augmen- tation is also beneficial when combined with pre-trained LM-based models, as it improves BERT-based joint intent and slot filling mod- els. 1 Introduction In task-oriented dialogue systems, a spoken lan- guage understanding component is responsible for parsing the user utterance into a semantic repre- sentation. This is often modeled as a semantic frame (Tur and De Mori, 2011), and typically in- volves slot filling and intent classification. For ex- ample, in the utterance “book in Southern Shores for 8 at Ariston Cafe”, the intent is BOOKING A RESTAURANT and the corresponding slot values and slot names are “Southern Shores” (CITY NAME), “8” (NUMBER OF PEOPLE), and “Ariston Cafe” (RESTAURANT NAME). Although neural-based models (Qin et al., 2019; Goo et al., 2018; Mesnil et al., 2015) have achieved stellar performance in slot filling (SF) and intent classification (IC), their performance depend on the availability of large labeled datasets. Consequently, they suffer in data scarcity situations, which regu- larly happen when new domains are added to the system to support new functionalities. One of the methods proposed to alleviate data scarcity is data augmentation (DA), which aims to automatically increase the size of the training data by applying data transformations, ranging from sim- ple word substitution to sentence generation. Re- cently, DA has shown promising potential for sev- eral NLP tasks, including text classification (Wei and Zou, 2019; Wang and Yang, 2015), parsing (Sahin and Steedman, 2018; Vania et al., 2019), and machine translation (Fadaee et al., 2017). As for SF and IC, DA approaches typically generate syn- thetic utterances by leveraging Seq2Seq (Hou et al., 2018; Zhao et al., 2019; Kurata et al., 2016), Con- ditional VAE (Yoo et al., 2019), or pre-trained Nat- ural Language Generation (NLG) models (Peng et al., 2020). Such approaches make use of in-domain data, and are relatively heavyweight, as they require training neural models, which may involve several phases to generate, filter, and rank the produced aug- mented data, thus requiring more computation time. It is also relatively challenging for deep learning- based models to generate semantically preserving synthetic utterances in limited data settings. In this paper, we show that lightweight augmen- tation, a set of simple DA methods that produce ut- terance variations, is very effective …

Content truncated to 500 words