Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1381–1393 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 1381 Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network Yutai Hou1, Wanxiang Che1∗, Yongkui Lai1, Zhihan Zhou3, Yijia Liu2, Han Liu3, Ting Liu1 1Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology 2Alibaba Group 3Department of Computer Science, Northwestern University {ythou, car, yklai, tliu}@ir.hit.edu.cn, oneplus.lau@gmail.com zhihanzhou2020@u.northwestern.edu, hanliu@northwestern.edu Abstract In this paper, we explore the slot tagging with only a few labeled support sentences (a.k.a. few-shot). Few-shot slot tagging faces a unique challenge compared to the other few- shot classification problems as it calls for mod- eling the dependencies between labels. But it is hard to apply previously learned label depen- dencies to an unseen domain, due to the dis- crepancy of label sets. To tackle this, we intro- duce a collapsed dependency transfer mecha- nism into the conditional random field (CRF) to transfer abstract label dependency patterns as transition scores. In the few-shot setting, the emission score of CRF can be calculated as a word’s similarity to the representation of each label. To calculate such similarity, we propose a Label-enhanced Task-Adaptive Pro- jection Network (L-TapNet) based on the state- of-the-art few-shot classification model – Tap- Net, by leveraging label name semantics in rep- resenting labels. Experimental results show that our model significantly outperforms the strongest few-shot learning baseline by 14.64 F1 scores in the one-shot setting.1 1 Introduction Slot tagging (Tur and De Mori, 2011), a key mod- ule in the task-oriented dialogue system (Young et al., 2013), is usually formulated as a sequence labeling problem (Sarikaya et al., 2016). Slot tag- ging faces the rapid changing of domains, and the labeled data is usually scarce for new domains with only a few samples. Few-shot learning technique (Miller et al., 2000; Fei-Fei et al., 2006; Lake et al., 2015; Vinyals et al., 2016) is appealing in this sce- nario since it learns the model that borrows the prior experience from old domains and adapts to new domains quickly with only very few examples (usually one or two examples for each class). ∗Corresponding author. 1Code is available at: https://github.com/ AtmaHou/FewShotTagging it O B-time O will Emission Score Transition Score Label-enhanced TapNet Collapsed Dependency Transfer B-weather rain tonight Few Support Examples will it snow next friday [O] [O] [B-weather] [B-time] [I-time] Is it strong wind outside [O][O] [B-weather] [I-weather] [O] 𝒙(𝟏) 𝒙���(𝟏) 𝒙(𝟏���) 𝒙���(𝟏���) 𝒙 𝒙��� Proposed CRF Framework 𝒙: will it rain tonight Label Semantics Time Weather Begin Ordinary [O] [B-weather] [I-weather] [B-time] [I-time] Label Name Semantics Query sentence Weather Time Inner Begin Inner Figure 1: Our few-shot CRF framework for slot tagging. Previous few-shot learning studies mainly fo- cused on classification problems, which have been widely explored with similarity-based methods (Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018; Yan et al., 2018; Yu et al., 2018). The ba- sic idea of these methods is classifying …

Content truncated to 500 words