Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9474–9480 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9474 Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling Liwen Wang1∗, Xuefeng Li1∗, Jiachi Liu1, Keqing He2, Yuanmeng Yan1, Weiran Xu1 1Beijing University of Posts and Telecommunications, Beijing, China 2Meituan Group, Beijing, China {w_liwen,lixuefeng,ljc1997,yanyuanmeng,xuweiran}@bupt.edu.cn {hekeqing}@meituan.com Abstract Zero-shot cross-domain slot filling alleviates the data dependence in the case of data scarcity in the target domain, which has aroused exten- sive research. However, as most of the exist- ing methods do not achieve effective knowl- edge transfer to the target domain, they just fit the distribution of the seen slot and show poor performance on unseen slot in the target domain. To solve this, we propose a novel ap- proach based on prototypical contrastive learn- ing with a dynamic label confusion strategy for zero-shot slot filling. The prototypical con- trastive learning aims to reconstruct the seman- tic constraints of labels, and we introduce the label confusion strategy to establish the label dependence between the source domains and the target domain on-the-fly. Experimental re- sults show that our model achieves significant improvement on the unseen slots, while also set new state-of-the-arts on slot filling task.1 1 Introduction Slot filling, as an important part of the task-oriented dialogue system, is mainly used to extract spe- cific information in user utterances. Traditional supervised methods have shown remarkable per- formance in slot filling tasks (Liu and Lane, 2016; Goo et al., 2018; E et al., 2019; He et al., 2020b; Wu et al., 2020; He et al., 2020a; Oguz and Vu, 2020; Qin et al., 2020), but they require a large amount of domain-specific labeled data. Recently, there has been increasing interest in the zero-shot cross-domain slot filling task, whose goal is to identify unseen slots in the target domain with- out any labeled data. Previous methods can be clas- sified into two types: one-stage and two-stage. In the one-stage framework, (Bapna et al., 2017; Shah ∗The first two authors contribute equally. Weiran Xu is the corresponding author. 1Our source code is available at: https://github. com/W-lw/PCLC CT RZT Coach Ours Models 0 10 20 30 40 50 60 70 Slot F1 1.8671 51.4324 7.1412 43.84 2.8977 55.8845 16.5671 62.8123 Unseen Slots Seen Slots (a) Performance of some zero-shot cross-domain slot fill- ing models on seen and unseen slots in GetWeather domain in SNIPS. Original Label Semantic Space Refined Label Semantic Space Dynamic Refinement Source Domain Cross-domain shared Target Domain Colors of Slot Prototypes : (b) The left part is the original distribution, where the slot prototypes of the source domain and the target domain are chaotic and lack constraint relations. The right part is the distribution after refinement, where slot prototypes of the source domain and target domain are separated. Figure 1: (a) Performance of the models in GetWeather domain; (b) a demonstration of slot prototypes refine- ment process; et al., 2019b; Lee and Jha, …

Content truncated to 500 words