ASSIST: Towards Label Noise-Robust Dialogue State Tracking

Findings of the Association for Computational Linguistics: ACL 2022, pages 2719 - 2731 May 22-27, 2022 c⃝2022 Association for Computational Linguistics ASSIST: Towards Label Noise-Robust Dialogue State Tracking Fanghua Ye and Yue Feng and Emine Yilmaz University College London London, UK {fanghua.ye.19, yue.feng.20, emine.yilmaz}@ucl.ac.uk Abstract The MultiWOZ 2.0 dataset has greatly boosted the research on dialogue state tracking (DST). However, substantial noise has been discov- ered in its state annotations. Such noise brings about huge challenges for training DST mod- els robustly. Although several refined versions, including MultiWOZ 2.1-2.4, have been pub- lished recently, there are still lots of noisy la- bels, especially in the training set. Besides, it is costly to rectify all the problematic annota- tions. In this paper, instead of improving the annotation quality further, we propose a gen- eral framework, named ASSIST (lAbel noiSe- robuSt dIalogue State Tracking), to train DST models robustly from noisy labels. ASSIST first generates pseudo labels for each sample in the training set by using an auxiliary model trained on a small clean dataset, then puts the generated pseudo labels and vanilla noisy la- bels together to train the primary model. We show the validity of ASSIST theoretically. Ex- perimental results also demonstrate that AS- SIST improves the joint goal accuracy of DST by up to 28.16% on MultiWOZ 2.0 and 8.41% on MultiWOZ 2.4, compared to using only the vanilla noisy labels. 1 Introduction Task-oriented dialogue systems play an important role in helping users accomplish a variety of tasks through verbal interactions (Young et al., 2013; Gao et al., 2019). Dialogue state tracking (DST) is an essential component of the dialogue manager in pipeline-based task-oriented dialogue systems. It aims to keep track of users’ intentions at each turn of the conversation (Mrkši´c et al., 2017). The state information indicates the progress of the conversa- tion and is leveraged to determine the next system action and generate the next system response (Chen et al., 2017). As shown in Figure 1, the dialogue state is typically represented as a set of (slot, value) pairs (Williams et al., 2014; Henderson et al., 2014). Hi, how may I help you? I need to book a room at autumn house. Definitely, for how many people and how many nights? Just me, 3 nights. Can you also give me information on the vue cinema? Sure. It is in the city centre, and the phone number is 08451962320. Thanks for your help. That’s all I need. (hotel-name, autumn house) (hotel-name, autumn house) (hotel-book people, 1) (hotel-book stay, 3) (attraction-name, vue cinema) (hotel-name, autumn house) (hotel-book people, 1) (hotel-book stay, 3) (attraction-name, vue cinema) Dialogue Context Dialogue State Figure 1: An example dialogue spanning two domains. On the left is the dialogue context with system respon- ses shown in orange and user utterances in green. The dialogue state at each turn is presented on the right. Therefore, the problem of DST is defined as extract- ing the values for all slots from the dialogue context at each turn …

Content truncated to 500 words