Few Shot Dialogue State Tracking using Meta-learning

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1730–1739 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1730 Few Shot Dialogue State Tracking using Meta-learning Saket Dingliwal1,2 Bill Gao1 Sanchit Agarwal1 Chien-Wei Lin1 Tagyoung Chung1 Dilek Hakkani-T¨ur1 1Amazon Alexa AI 2Carnegie Mellon University {skdin,shuyag,agsanchi,chienwel,tagyoung,hakkanit}@amazon.com Abstract Dialogue State Tracking (DST) forms a core component of automated chatbot based sys- tems designed for specific goals like hotel, taxi reservation, tourist information etc. With the increasing need to deploy such systems in new domains, solving the problem of zero/few- shot DST has become necessary. There has been a rising trend for learning to transfer knowledge from resource-rich domains to un- known domains with minimal need for addi- tional data. In this work, we explore the merits of meta-learning algorithms for this transfer and hence, propose a meta-learner D- REPTILE specific to the DST problem. With extensive experimentation, we provide clear evidence of benefits over conventional ap- proaches across different domains, methods, base models and datasets with significant (5- 25%) improvement over the baseline in low- data setting. Our proposed meta-learner is ag- nostic of the underlying model and hence any existing state-of-the-art DST system can im- prove its performance on unknown domains using our training strategy. 1 Introduction Task-Oriented Dialogue (TOD) systems are auto- mated conversational agents built for a specific goal (for example hotel reservation). Many businesses from wide-variety of domains (like hotel, restau- rant, car-rental, payments etc) have adopted these systems to cut down their cost on customer sup- port services. Almost all such systems have a Dia- logue State Tracking (DST) module which keeps track of values for some predefined domain-specific slots (example hotel-name, restaurant-rating etc) after every turn of utterances from user and system. These values are then used by Natural Language Generator (NLG) to generate system responses and fulfill the user goals. Many of the recent works (Wu et al., 2019; Zhang et al., 2019; Goel et al., 2019; Heck et al., 2020) have proposed various neural models that achieve good performance for the task but are data hungry in general. Therefore, adapting to a new unknown domain (target domain) requires large amounts of domain-specific annotations limiting their use. However, given a wide range of prac- tical applications, there has been a recent inter- est in data-efficient approaches. Lee et al. (2019), Gao et al. (2020) used transformer (Vaswani et al., 2017) based models which significantly reduce data dependence. Further, Gao et al. (2020) model the problem as machine reading comprehension task and benefit from its readily available external datasets and methods. Wu et al. (2019) were first to propose transferring knowledge from one domain to another. Since, many domains like restaurant and hotel share a lot of common slots like name, area, rating, etc and hence such a transfer proved to be effective for a low-resource domain. More recently, Campagna et al. (2020) aimed at zero-shot DST using synthetic data generation for the target domain imitating data …

Content truncated to 500 words