Few-shot Learning for Slot Tagging with Attentive Relational Network

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1566–1572 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1566 Few-shot Learning for Slot Tagging with Attentive Relational Network Cennet Oguz1 Ngoc Thang Vu2 1Multilinguality and Language Technology, German Research Center for Artificial Intelligence 2Institute for Natural Language Processing, University of Stuttgart cennet.oguz@dfki.de thangvu@ims.uni-stuttgart.de Abstract Metric-based learning is a well-known fam- ily of methods for few-shot learning, espe- cially in computer vision. Recently, they have been used in many natural language pro- cessing applications but not for slot tagging. In this paper, we explore metric-based learn- ing methods in the slot tagging task and pro- pose a novel metric-based learning architec- ture - Attentive Relational Network. Our pro- posed method extends relation networks, mak- ing them more suitable for natural language processing applications in general, by lever- aging pretrained contextual embeddings such as ELMO and BERT and by using attention mechanism. The results on SNIPS data show that our proposed method outperforms other state of the art metric-based learning methods. 1 Introduction Neural networks have been successfully utilized in natural language processing (NLP) applications with a large amount of hand-labeled data whereas they suffer a persistent challenge of low-resource. The approach of learning with few samples, known as few-shot learning - a branch of meta-learning (learn to learn) - has recently been popularized (Fei-Fei et al., 2006; Ravi and Larochelle, 2016; Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018) in computer vision. Recently, few-shot learn- ing has also been applied to NLP tasks, e.g. natural language understanding (Dou et al., 2019), text classification (Jiang et al., 2018; Rios and Kavu- luru, 2018; Gao et al., 2019; Geng et al., 2019), machine translation (Gu et al., 2018) and relation classification (Obamuyide and Vlachos, 2019). In the slot tagging task, we aim at predicting task- specific values (e.g. artist, time) for slots (place- holders) in user utterances. Oguz and Vu (2020) propose a two-stage modeling approach to exploit domain-agnostic features to tackle low-resource do- main challenges. Besides, the other state of the art techniques e.g. based on external memory (Peng and Yao, 2015), ranking loss (Vu et al., 2016), en- coder (Kurata et al., 2016), and attention (Zhu and Yu, 2017) have achieved promising results with a wide range of neural networks methods. However as many other NLP applications, the low-resource issue is a tremendous challenge for slot tagging in new domains, although labeled sam- ples exist in related domains. Many studies have recently proposed to overcome this low-resource challenge using different techniques, e.g. multi- task modeling (Jaech et al., 2016), adversarial train- ing (Kim et al., 2017), and pointer networks (Zhai et al., 2017). In addition, studies like zero-shot learning has influenced the studies of the domain scaling problem for slots prediction (Bapna et al., 2017), eliminating the need of labeled examples for transferring reusable concepts (Zhu and Yu, 2018; Lee and Jha, 2019), and conveying the domain- agnostic …

Content truncated to 500 words