Zero-shot Cross-lingual Dialogue Systems with Transferable Latent Variables

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1297–1303, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1297 Zero-shot Cross-lingual Dialogue Systems with Transferable Latent Variables Zihan Liu, Jamin Shin, Yan Xu, Genta Indra Winata, Peng Xu, Andrea Madotto, Pascale Fung Center for Artificial Intelligence Research (CAiRE) Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong {zliucr,jay.shin,yxucb,giwinata,pxuab}@connect.ust.hk, pascale@ece.ust.hk Abstract Despite the surging demands for multilin- gual task-oriented dialog systems (e.g., Alexa, Google Home), there has been less research done in multilingual or cross-lingual scenar- ios. Hence, we propose a zero-shot adapta- tion of task-oriented dialogue system to low- resource languages. To tackle this challenge, we first use a set of very few parallel word pairs to refine the aligned cross-lingual word- level representations. We then employ a latent variable model to cope with the variance of similar sentences across different languages, which is induced by imperfect cross-lingual alignments and inherent differences in lan- guages. Finally, the experimental results show that even though we utilize much less external resources, our model achieves better adapta- tion performance for natural language under- standing task (i.e., the intent detection and slot filling) compared to the current state-of-the-art model in the zero-shot scenario. 1 Introduction Task-oriented dialogue systems have been widely adopted in the industry (e.g., Amazon Alexa, Google Home, Apple Siri, Microsoft Cortana) as a virtual agent to tend to the needs of the users. However, these agents have mostly been trained with the monolingual dataset that is often expen- sive to build or acquire. In order to cope with the scarcity of low-resource language dialogue data, we are motivated to look into cross-lingual dia- logue systems which can adapt with very little or no training data in the target language. This task of zero-shot adaptation of dialogue systems to different languages is relatively new and has not been explored thoroughly enough yet. The main approach of previous work (Upadhyay et al., 2018; Chen et al., 2018; Schuster et al., 2019) in this task is using aligned cross-lingual Figure 1: (a) The aligned word embeddings. (b) Re- fined mapping W with seed dictionary; green circle represents close words in different languages. word embeddings between source and target lan- guages. However, this method suffers from im- perfect alignments between the source and target language embeddings. This can be attributed not only to the noise in aligning two different em- beddings, but also to the inherent discrepancies in different languages such as Thai and English which come from entirely different roots. To ad- dress such variance in the alignment, we turn to probabilistic modeling with latent variables as it has been successfully used in several recent task- oriented dialogue systems (Wen et al., 2017; Zhao et al., 2017, 2018; Zhao and Eskenazi, 2018; Le et al., 2018). However, we notice that naively using latent variables does not help …

Content truncated to 500 words