Few-shot Intent Classification and Slot Filling with Retrieved Examples

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 734–749 June 6–11, 2021. ©2021 Association for Computational Linguistics 734 Few-shot Intent Classification and Slot Filling with Retrieved Examples Dian Yu∗ University of California, Davis dianyu@ucdavis.edu Luheng He Google Research luheng@google.com Yuan Zhang Google Research zhangyua@google.com Xinya Du∗ Cornell University xd75@cornell.edu Panupong Pasupat Google Research ppasupat@google.com Qi Li Google Assistant qilqil@google.com Abstract Few-shot learning arises in important practical scenarios, such as when a natural language un- derstanding system needs to learn new seman- tic labels for an emerging, resource-scarce do- main. In this paper, we explore retrieval-based methods for intent classification and slot fill- ing tasks in few-shot settings. Retrieval-based methods make predictions based on labeled ex- amples in the retrieval index that are similar to the input, and thus can adapt to new do- mains simply by changing the index without having to retrain the model. However, it is non- trivial to apply such methods on tasks with a complex label space like slot filling. To this end, we propose a span-level retrieval method that learns similar contextualized representa- tions for spans with the same label via a novel batch-softmax objective. At inference time, we use the labels of the retrieved spans to con- struct the final structure with the highest ag- gregated score. Our method outperforms pre- vious systems in various few-shot settings on the CLINC and SNIPS benchmarks. 1 Introduction Few-shot learning is a crucial problem for practical language understanding applications. In the few- shot setting, the model (typically trained on source domains with abundant data) needs to adapt to a set of unseen labels in the target domain with only a few examples. For instance, when developers introduce a new product feature, a query under- standing model has to learn new semantic labels from a small dataset they manage to collect. Few-shot learning is challenging due to the im- balance in the amount of data between the source and target domains. Traditional classification meth- ods, even with the recent advancement of pre- trained language models (Peters et al., 2018; Devlin et al., 2019), could suffer from over-fitting (Snell ∗work done during internship at Google Research et al., 2017; Triantafillou et al., 2019) or catas- trophic forgetting (Wu et al., 2019) when incorpo- rating the data-scarce target domain. On the other hand, metric learning methods (Weinberger et al., 2006; Vinyals et al., 2016; Snell et al., 2017) have been shown to work well in few-shot scenarios. These methods are based on modeling similarity be- tween inputs, effectively allowing the model to be decoupled from the semantics of the output space. For example, a model would learn that the utter- ance “I’d like to book a table at black horse tavern at 7 pm” (from Figure 1) is similar to “make me a reservation at 8” and thus are likely to have similar semantic representations, even without knowing the semantic schema in use. Unlike learning out- put labels, …

Content truncated to 500 words