N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking

Findings of the Association for Computational Linguistics: ACL 2022, pages 1659 - 1671 May 22-27, 2022 c⃝2022 Association for Computational Linguistics N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking Taha Aksu†‡∗,, Zhengyuan Liu‡§, Min-Yen Kan†, Nancy F. Chen‡ † National University of Singapore ‡ Institute for Infocomm Research, A*STAR § CNRS@CREATE1 *taksu@u.nus.edu Abstract Augmentation of task-oriented dialogues has followed standard methods used for plain-text such as back-translation, word-level manipu- lation, and paraphrasing despite its richly an- notated structure. In this work, we introduce an augmentation framework that utilizes belief state annotations to match turns from various dialogues and form new synthetic dialogues in a bottom-up manner. Unlike other augmenta- tion strategies, it operates with as few as five examples. Our augmentation strategy yields significant improvements when both adapting a DST model to a new domain, and when adapting a language model to the DST task, on evaluations with TRADE and TOD-BERT models. Further analysis shows that our model performs better on seen values during train- ing, and it is also more robust to unseen val- ues. We conclude that exploiting belief state annotations enhances dialogue augmentation and results in improved models in n-shot train- ing scenarios. 1 Introduction Task-oriented dialogue (TOD) agents are the next- generation user interface and are slated to replace browsing static websites. However, a key bottle- neck in fielding such agents practically concerns adapting to new domains with few available data. In the light of this dependency in ample amounts of annotated data, data augmentation is growing in importance (Feng et al., 2021). Most augmenta- tion methods in natural language processing (NLP) target written forms of text — passages, news arti- cles, etc. — which operate with word- or sentence- level permutations of the original text data, syn- thesizing new text (Liu et al., 2020; Wei and Zou, 2019; Yu et al., 2018; Xie et al., 2017; Kobayashi, 2018). These methods do not exploit the structure 1CNRS@CREATE LTD, 1 Create Way, #08-01 CREATE Tower, Singapore 138602 Figure 1: Scenario with two dialogues from train booking domain. Dialogue snippets, SA&SB, have the same dialogue function and the new dialogue created by replacing them and inserting proper slot values is still coherent end to end. of conversational data in its entirety. We study aug- menting task-oriented dialogues, a specific form of conversational data. A TOD is a form of conversation where the aim is to accomplish a task through exchanges between a user and an agent, accounting for the user’s pref- erences. Within TOD, dialogue state tracking (DST) is a fundamental task, which aims to detect these preferences in a given dialogue. For this task, each pair of utterances in a dialogue is annotated with slot-label and slot-value pairs (cf. Figure 1: train- destination: “Cambridge”) and a belief state. Here, a belief state can be equated as an attribute–value store that gives the final values of each slot label (attribute) after an utterance. There have been several attempts to augment conversational data in the literature. Quan and Xiong …

Content truncated to 500 words