HCLD: A Hierarchical Framework for Zero-shot Cross-lingual Dialogue System

Proceedings of the 29th International Conference on Computational Linguistics, pages 4492–4498 October 12–17, 2022. 4492 HCLD: A Hierarchical Framework for Zero-shot Cross-lingual Dialogue System Zhanyu Ma1,2,3 Jian Ye1,2,3∗ Xurui Yang1,2,3 Jianfeng Liu1,2,3 Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China1 University of Chinese Academy of Sciences2 Beijing Key Laboratory of Mobile Computing and Pervasive Device3 {mazhanyu21s,jye,yangxurui20g,liujianfeng18s}@ict.ac.cn Abstract Recently, many task-oriented dialogue systems need to serve users in different languages. How- ever, it is time-consuming to collect enough data of each language for training. Thus, zero- shot adaptation of cross-lingual task-oriented dialog systems has been studied. Most of ex- isting methods consider the word-level align- ments to conduct two main tasks for task- oriented dialogue system, i.e., intent detection and slot filling, and they rarely explore the de- pendency relations among these two tasks. In this paper, we propose a hierarchical frame- work to classify the pre-defined intents in the high-level and fulfill slot filling under the guid- ance of intent in the low-level. Particularly, we incorporate sentence-level alignment among different languages to enhance the performance of intent detection. The extensive experiments report that our proposed method achieves the SOTA performance on the public task-oriented dialog dataset. 1 Introduction Natural language understanding (NLU) plays a significant role in task-oriented dialogue systems, which is aimed to parse dialog utterances by iden- tifying user’s intent and the arguments of the in- tent (Hou et al., 2021; van der Goot et al., 2021). These two tasks are known as intent detection and slot filling respectively. For instance, in the utter- ance “Next week’s forecast” as shown in Table 1, the user intent is to query about the weather cir- cumstances and the time argument of the query is “next week”. Recently, many neural models have been proposed to jointly train these two tasks by considering the intent detection and the slot filling as sentence classification and sequence labeling task (Krishnan et al., 2021; Cao et al., 2020). However, most works focus on monolingual datasets which are expensive to build. Furthermore, ∗ Corresponding author. Sentence Next week’s forecast Slots B-datetime I-datetime B-weather/noun Intents weather/find Table 1: An example of a sentence with slots and intents annotations from the zero-shot NLU dataset. some dialogue systems, e.g., Google Home and Apple Siri, need to serve numerous users around the world in different languages and might be faced with the scarcity of dialogue data in certain lan- guages. Thus, it’s motivated to build cross-lingual dialogue systems that enable zero-shot adaptation from a high-resource language to a low-resource language without any training data in the target language. Specifically, some models (Upadhyay et al., 2018; Chen et al., 2018; Schuster et al., 2019) used cross-lingual pretrained embeddings to bridge different languages. Some approaches like Liu et al. (2019) adopted a small vocabulary of trans- lated word-pairs to align cross-lingual embeddings by bilingual projection and alleviate the inherent discrepancies among different languages. Many existing methods (Liu et al., 2020; Qin et al., 2020) utilized pretrained cross-lingual language model (PXLM), …

Content truncated to 500 words