Contrastive Zero-Shot Learning for Cross-Domain Slot Filling with Adversarial Attack

Proceedings of the 28th International Conference on Computational Linguistics, pages 1461–1467 Barcelona, Spain (Online), December 8-13, 2020 1461 Contrastive Zero-Shot Learning for Cross-Domain Slot Filling with Adversarial Attack Keqing He1∗, Jinchao Zhang2, Yuanmeng Yan1, Weiran Xu1, Cheng Niu2, Jie Zhou2 1Beijing University of Posts and Telecommunications, Beijing, China 2Pattern Recognition Center, WeChat AI, Tencent {kqin,yanyuanmeng,xuweiran}@bupt.edu.cn {dayerzhang,withtomzhou}@tencent.com {chengniu}@gmail.com Abstract Zero-shot slot filling has widely arisen to cope with data scarcity in target domains. However, previous approaches often ignore constraints between slot value representation and related slot description representation in the latent space and lack enough model robustness. In this paper, we propose a Contrastive Zero-Shot Learning with Adversarial Attack (CZSL-Adv) method for the cross-domain slot filling. The contrastive loss aims to map slot value contextual represen- tations to the corresponding slot description representations. And we introduce an adversarial attack training strategy to improve model robustness. Experimental results show that our model significantly outperforms state-of-the-art baselines under both zero-shot and few-shot settings. 1 Introduction The slot filling task in the goal-oriented dialog system aims to identify task-related slot types in certain domains for understanding user utterances. Traditional supervised slot filling models (Liu and Lane, 2015; Liu and Lane, 2016; Goo et al., 2018; Haihong et al., 2019; He et al., 2020a; He et al., 2020b) have made great achievements. However, these models require massive amounts of labeled data for a new domain, hindering the rapid development of new tasks. To address the data-intensiveness problem, domain adaptation approaches (Bapna et al., 2017; Lee and Jha, 2019; Shah et al., 2019; Obeidat et al., 2019; Liu et al., 2020b; He et al., 2020c) have been successfully applied. In this paper, we focus on zero-shot cross-domain transfer learning which leverages knowledge learned in the source domains and adapts the models to the target domain without labeled training samples in the target domain. The main challenge of zero-shot slot filling is to identify unseen slot types without any supervision signals in the target domain. Typically, the previous methods rely on slot descriptions or example values to bootstrap to new slots by capturing the semantic relationship between slot descriptions and input tokens. These methods can be classified into two categories: one-stage and two-stage. (Bapna et al., 2017; Lee and Jha, 2019; Shah et al., 2019) conduct one-stage slot filling individually for each slot type. They first generate word-level representations, then interact with the representation of each slot type description in semantic space. Finally, the predictions are independent for each slot type based on the fused features. The main drawback is the multiple prediction problem where a word may be predicted as multiple slot types. In contrast, (Liu et al., 2020a; Liu et al., 2020b) propose a two-stage slot filling framework. They first predict whether the tokens are slot entities or not by a BIO 3-way classifier, then identify their specific slot types based on slot type descriptions. Although the two-stage framework helps learn the general pattern of slot entities, it can’t directly leverage auxiliary description information to …

Content truncated to 500 words