AISFG: Abundant Information Slot Filling Generator

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4180 - 4187 July 10-15, 2022 ©2022 Association for Computational Linguistics AISFG: Abundant Information Slot Filling Generator Yang Yan∗, Junda Ye∗, Zhongbao Zhang, Liwen Wang Beijing University of Posts and Telecommunications, Beijing, China {yanyang42,jundaye,zhongbaozb,w_liwen}@bupt.edu.cn Abstract As an essential component of task-oriented di- alogue systems, slot filling requires enormous labeled training data in a certain domain. How- ever, in most cases, there is little or no tar- get domain training data is available in the training stage. Thus, cross-domain slot filling has to cope with the data scarcity problem by zero/few-shot learning. Previous researches on zero/few-shot cross-domain slot filling fo- cus on slot descriptions and examples while ignoring the slot type ambiguity and example ambiguity issues. To address these problems, we propose Abundant Information Slot Filling Generator (AISFG), a generative model with a novel query template that incorporates do- main descriptions, slot descriptions, and exam- ples with context. Experimental results show that our model outperforms state-of-the-art ap- proaches in zero/few-shot slot filling task.1 1 Introduction Slot filling is a critical part of downstream tasks in natural language understanding (NLU) such as di- alogue systems. Recently, some supervised slot filling models have achieved state-of-the-art re- sults within deep learning (Mesnil et al., 2013; Yao et al., 2013; Louvan and Magnini, 2018; Kim et al., 2019).Nonetheless, these methods have a strong dependency on the domain-specific labels, which is not capable of transferring to new domains that always contain little or no data. To alleviate the problem of resource gap between source and target domains, cross-domain zero-shot has become an important research direction in slot filling. However, most researches on slot filling utilize token-level classification frameworks, which means they either convert it to a BIO tag labeling task (Shah et al., 2019; Bapna et al., 2017; Du et al., 2021) or predict the start and end position as *Contributed equally. 1The source code is available at https://github. com/realyanyang/AISFG. QA task in the sentence (Du et al., 2021; Yu et al., 2021a), to extract spans. Recently, prompt learning methods reformulate the downstream tasks to a similar form with the pre-training tasks, which can fully utilize the knowledge encoded in PLMs and improve the performance of the downstream tasks in the scenarios of data scarcity. Motivated by this, we consider making the slot filling task consistent with PLMs pre-training tasks. In this paper, we propose a generative template- based zero-shot slot filling framework named Abun- dant Information Slot Filling Generator (AISFG), which utilizes pre-trained generative model as the backbone and generates responses in natural lan- guage style. Thus, the slot filling task is consistent with the pre-training task of the PLM. In particu- lar, we notice that shared cross-domain slot types sometimes refer to totally different entities across different domains. For example, in the SNIPS dataset (Coucke et al., 2018), domains RateBook and SearchScreeningEvent share a common slot type object_type, but it refers to book type …

Content truncated to 500 words