Pre-trained Token-replaced Detection Model as Few-shot Learner

Proceedings of the 29th International Conference on Computational Linguistics, pages 355–366 October 12–17, 2022. 355 Schema Encoding for Transferable Dialogue State Tracking Hyunmin Jeon Computer Science and Engineering POSTECH, Pohang, South Korea jhm9507@postech.ac.kr Gary Geunbae Lee Computer Science and Engineering Graduate School of Artificial Intelligence POSTECH, Pohang, South Korea gblee@postech.ac.kr Abstract Dialogue state tracking (DST) is an essential sub-task for task-oriented dialogue systems. Recent work has focused on deep neural mod- els for DST. However, the neural models re- quire a large dataset for training. Furthermore, applying them to another domain needs a new dataset because the neural models are gener- ally trained to imitate the given dataset. In this paper, we propose Schema Encoding for Transferable Dialogue State Tracking (SET- DST), which is a neural DST method for effec- tive transfer to new domains. Transferable DST could assist developments of dialogue systems even with few dataset on target domains. We use a schema encoder not just to imitate the dataset but to comprehend the schema of the dataset. We aim to transfer the model to new domains by encoding new schemas and using them for DST on multi-domain settings. As a result, SET-DST improved the joint accuracy by 1.46 points on MultiWOZ 2.1. 1 Introduction The objective of task-oriented dialogue systems is to help users achieve their goals by conversations. Dialogue state tracking (DST) is the essential sub- task for the systems to perform the purpose. Users may deliver the details of their goals to the sys- tems during the conversations, e.g., what kind of food they want the restaurant to serve and at what price level they want to book the hotel. Thus, the systems should exactly catch the details from utter- ances. They should also communicate with other systems by using APIs to achieve users’ goals, e.g., to search restaurants and to reserve hotels. The goal of DST is not only to classify the users’ intents but also to fill the details into predefined templates that are used to call APIs. Recent work has used deep neural networks for DST with supervised learning. They have im- proved the accuracy of DST; however, they require a large dataset for training. Furthermore, they need a new dataset to be trained on another domain. Un- fortunately, the large dataset for training a DST model is not easy to be developed in real world. The motivation of supervised learning is to make deep neural networks imitate humans. But, they ac- tually imitate the given datasets rather than humans. Someones who have performed hotel reservation work could easily perform restaurant reservation work if some guidelines are provided, but neural models may have to be trained on a new dataset of the restaurant domain. The difference between humans and neural models is that humans can learn how to read guidelines and to apply the guidelines to their work. This is why transfer learning is im- portant to train neural models on new domains. In this paper, we propose Schema Encoding for Transferable Dialogue …

Content truncated to 500 words