Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 902–915 May 2-6, 2023 ©2023 Association for Computational Linguistics Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues Mehrad Moradshahi1 Victoria Tsai2 Giovanni Campagna1 Monica S. Lam1 1 Stanford University, Stanford, CA 2 Meta, Menlo Park, CA {mehrad,gcampagn,lam}@cs.stanford.edu {vitsai}@meta.com Abstract Robust state tracking for task-oriented dia- logue systems currently remains restricted to a few popular languages. This paper shows that given a large-scale dialogue data set in one lan- guage, we can automatically produce an effec- tive semantic parser for other languages using machine translation. We propose automatic translation of dialogue datasets with alignment to ensure faithful translation of slot values and eliminate costly human supervision used in previous benchmarks. We also propose a new contextual semantic parsing model, which en- codes the formal slots and values, and only the last agent and user utterances. We show that the succinct representation reduces the com- pounding effect of translation errors, without harming the accuracy in practice. We evaluate our approach on several dialogue state tracking benchmarks. On RiSAWOZ, CrossWOZ, CrossWOZ-EN, and MultiWOZ- ZH datasets we improve the state of the art by 11%, 17%, 20%, and 0.3% in joint goal accu- racy. We present a comprehensive error anal- ysis for all three datasets showing erroneous annotations can lead to misguided judgments on the quality of the model. Finally, we present RiSAWOZ English and German datasets, created using our transla- tion methodology. On these datasets, accu- racy is within 11% of the original showing that high-accuracy multilingual dialogue datasets are possible without relying on expensive hu- man annotations. We release our datasets and software open source.1 1 Introduction Tremendous effort has gone into the research and development of task-oriented dialogue agents for English and a few other major languages in recent years. A methodology that can transfer the effort to 1https://github.com/stanford-oval/ dialogues other languages automatically will greatly benefit the large population of speakers of the many other languages in the world. Underlying an effective TOD agent is dialogue state tracking, the task of predicting a formal rep- resentation of the conversation sufficient for the dialogue agent to reply, in the form of slots and values. However, DST currently remains restricted to a few popular languages (Razumovskaia et al., 2021). Traditional DST agents require large hand- annotated Wizard-of-Oz (Kelley, 1984) datasets for training, which are prohibitively labor-intensive to produce in most languages (Gunasekara et al., 2020). Large, multi-domain WOZ datasets are only available in English and Chinese (Quan et al., 2020; Ye et al., 2021a). The contributions of this paper are as follows: 1. We propose an automatic technique to build multilingual data sets using machine translation. Machine translation has been shown effective for localizing question-answering agents (Moradshahi et al., 2020). It shows that for open ontology datasets, we need to use an align- ment model to properly translate entities in the source language to entities in the target language. This paper shows that alignment is necessary even for …

Content truncated to 500 words