NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3704–3717 August 1–6, 2021. ©2021 Association for Computational Linguistics 3704 NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation Sungdong Kim1,2 Minsuk Chang1,2 Sang-Woo Lee1,2 NAVER AI Lab1 NAVER Clova2 {sungdong.kim, minsuk.chang, sang.woo.lee}@navercorp.com Abstract We propose NeuralWOZ, a novel dialogue collection framework that uses model-based dialogue simulation. NeuralWOZ has two pipelined models, Collector and Labeler. Col- lector generates dialogues from (1) user’s goal instructions, which are the user context and task constraints in natural language, and (2) system’s API call results, which is a list of possible query responses for user requests from the given knowledge base. Labeler an- notates the generated dialogue by formulating the annotation as a multiple-choice problem, in which the candidate labels are extracted from goal instructions and API call results. We demonstrate the effectiveness of the proposed method in the zero-shot domain transfer learn- ing for dialogue state tracking. In the evalua- tion, the synthetic dialogue corpus generated from NeuralWOZ achieves a new state-of-the- art with improvements of 4.4% point joint goal accuracy on average across domains, and im- provements of 5.7% point of zero-shot cover- age against the MultiWOZ 2.1 dataset.1 1 Introduction For a task-oriented dialogue system to be scalable, the dialogue system needs to be able to quickly adapt and expand to new scenarios and domains. However, the cost and effort in collecting and an- notating an expanding dataset is not only labor- intensive but also proportional to the size and vari- ety of the unseen scenarios. There are three types of dialogue system expan- sions. (1) The simplest expansion is the addition of new instances in the knowledge base (KB) under the identical schema. For example, the addition of newly opened restaurants in the KB of restaurant domain falls under this category. (2) A slightly more complicated expansion involves modifica- tions to the KB schema, and possibly the related 1The code is available at github.com/naver-ai/neuralwoz. Figure 1: Overview of NeuralWOZ. The NeuralWOZ takes goal instruction for the user side (U) and API call results for the system side (S) to synthesize dialogue. First, it generates dialogue from the inputs and then la- bels dialogue state (Bt) and active domain (Domaint) by turn t on the dialogue. instances. For example, additions of new constraint types to access the KB due to the change in needs of the user often require a restructuring of the KB. If a dialogue system built with only restaurant search in mind observes user’s requests about not only “restaurant location” and but also “traffic informa- tion” for navigating, the system now needs a new knowledge base including the additional different domain. (3) The most complex expansion is the one that expands across multiple domains. For ex- ample, imagine an already built dialogue system 3705 supported restaurant and hotel reservation domains, but now needs to expand to points of interest or other domains. It is difficult …

Content truncated to 500 words