Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2048–2061 July 9-14, 2023 ©2023 Association for Computational Linguistics Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking Qingyue Wang♠♣, Liang Ding♢, Yanan Cao♠∗, Yibing Zhan♢, Zheng Lin♠, Shi Wang♡, Dacheng Tao∇ and Li Guo♠ ♠ Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China ♣ School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China ♡ Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China ♢ JD Explore Academy, JD.com Inc, China ∇The University of Sydney, Australia {wangqingyue,caoyanan,linzheng,guoli}@iie.ac.cn, wangshi@ict.ac.cn {liangding.liam,zhanybjy,dacheng.tao}@gmail.com Abstract Zero-shot transfer learning for Dialogue State Tracking (DST) helps to handle a variety of task-oriented dialogue domains without the cost of collecting in-domain data. Existing works mainly study common data- or model- level augmentation methods to enhance the gen- eralization but fail to effectively decouple the semantics of samples, limiting the zero-shot performance of DST. In this paper, we present a simple and effective “divide, conquer and com- bine” solution, which explicitly disentangles the semantics of seen data, and leverages the performance and robustness with the mixture- of-experts mechanism. Specifically, we divide the seen data into semantically independent subsets and train corresponding experts, the newly unseen samples are mapped and inferred with mixture-of-experts with our designed en- semble inference. Extensive experiments on MultiWOZ2.1 upon the T5-Adapter show our schema significantly and consistently improves the zero-shot performance, achieving the SOTA on settings without external knowledge, with only 10M trainable parameters1. 1 Introduction Dialogue state tracking (DST) plays an impor- tant role in many task-oriented dialogue systems (Young et al., 2013). The goal of this task is to understand users’ needs and goals by exacting dia- logue states at each turn, which are typically in the form of a list of slot-value pairs (Wu et al., 2019). Accurate DST performance can help downstream applications such as dialogue management. However, collecting and annotating the dia- logue state is notoriously hard and expensive (Budzianowski et al., 2018). This problem becomes ∗Yanan Cao is the corresponding author. 1Code is freely available at: https://github.com/ qingyue2014/MoE4DST.git I would like a taxi from saint johns college to pizza hut fen ditton. I am traveling on Sunday from peterborough to cambridge. Semantics area Seen sample Book me some rooms for 6 people on Wednesday unseen sample Figure 1: An illustration of the semantics areas of seen data and perform inference on the newly unseen sample. For each sample, We omit previous turns and only show the current utterance from the user. pressing from single-domain to multi-domain sce- narios. To train a multi-domain DST model, di- alogue annotators need to indicate all slot-value pairs for each domain and turn. Therefore, tracking unseen slots in a new domain without any labels, i.e. zero-short prediction, is becoming an urgent demand for real-world deployments. To make the DST module more practical, e.g. robust to unseen domains, various methods have been developed to improve the zero-shot capacity …

Content truncated to 500 words