Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3687 - 3703 July 10-15, 2022 ©2022 Association for Computational Linguistics Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog Chia-Chien Hung1, Anne Lauscher2, Ivan Vuli´c3, Simone Paolo Ponzetto1 and Goran Glavaˇs4 1Data and Web Science Group, University of Mannheim, Germany 2MilaNLP, Bocconi University, Italy 3 LTL, University of Cambridge, UK 4 CAIDAS, University of W¨urzburg, Germany {chia-chien, simone}@informatik.uni-mannheim.de anne.lauscher@unibocconi.it iv250@cam.ac.uk goran.glavas@uni-wuerzburg.de Abstract Research on (multi-domain) task-oriented di- alog (TOD) has predominantly focused on the English language, primarily due to the shortage of robust TOD datasets in other lan- guages, preventing the systematic investiga- tion of cross-lingual transfer for this crucial NLP application area. In this work, we intro- duce MULTI2WOZ, a new multilingual multi- domain TOD dataset, derived from the well- established English dataset MULTIWOZ, that spans four typologically diverse languages: Chinese, German, Arabic, and Russian. In con- trast to concurrent efforts (Ding et al., 2021; Zuo et al., 2021), MULTI2WOZ contains gold- standard dialogs in target languages that are directly comparable with development and test portions of the English dataset, enabling reli- able and comparative estimates of cross-lingual transfer performance for TOD. We then intro- duce a new framework for multilingual conver- sational specialization of pretrained language models (PrLMs) that aims to facilitate cross- lingual transfer for arbitrary downstream TOD tasks. Using such conversational PrLMs spe- cialized for concrete target languages, we sys- tematically benchmark a number of zero-shot and few-shot cross-lingual transfer approaches on two standard TOD tasks: Dialog State Track- ing and Response Retrieval. Our experiments show that, in most setups, the best performance entails the combination of (i) conversational specialization in the target language and (ii) few-shot transfer for the concrete TOD task. Most importantly, we show that our conversa- tional specialization in the target language al- lows for an exceptionally sample-efficient few- shot transfer for downstream TOD tasks. 1 Introduction Task-oriented dialog (TOD) is arguably one of the most popular natural language processing (NLP) application areas (Yan et al., 2017; Henderson et al., 2019, inter alia), with more importance recently given to more realistic, and thus, multi-domain con- versations (Budzianowski et al., 2018; Ramadan et al., 2018), in which users may handle more than one task during the conversation, e.g., booking a taxi and making a reservation at a restaurant. Un- like many other NLP tasks (e.g., Hu et al., 2020; Liang et al., 2020; Ponti et al., 2020, inter alia), the progress towards multilingual multi-domain TOD has been hindered by the lack of sufficiently large and high-quality datasets in languages other than English (Budzianowski et al., 2018; Zang et al., 2020) and more recently, Chinese (Zhu et al., 2020). This lack can be attributed to the fact that creating TOD datasets for new languages from scratch or via translation of English datasets is significantly more expensive and time-consuming than for most other NLP tasks. However, the absence of multi- lingual datasets …

Content truncated to 500 words