Towards Zero-Shot Multilingual Transfer for Code-Switched Responses

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7551–7563 July 9-14, 2023 ©2023 Association for Computational Linguistics Towards Zero-Shot Multilingual Transfer for Code-Switched Responses Ting-Wei Wu1∗, Changsheng Zhao2, Ernie Chang2, Yangyang Shi2, Pierce Chuang2, Vikas Chandra2, Biing Juang1 1Georgia Institute of Technology 2Meta Reality Labs waynewu@gatech.edu, juang@ece.gatech.edu, {cszhao, erniecyc, yyshi, pichuang, vchandra}@meta.com Abstract Recent task-oriented dialog systems obtained great successes in building personal assistants for high resource language such as English, but extending these systems to a global audience is challenging due to the need for annotated data or machine translation systems in the target lan- guage. An alternative approach is to leverage existing data in a high-resource language to en- able cross-lingual transfer in low-resource lan- guage models. However, this type of transfer has not been widely explored in natural lan- guage response generation. In this research, we investigate the use of state-of-the-art multilin- gual models such as mBART and T5 to facil- itate zero-shot and few-shot transfer of code- switched responses. We propose a new adapter- based framework that allows for efficient trans- fer by learning jointly the task-specific, source and target language representations. Our frame- work is able to successfully transfer language knowledge even when the target language cor- pus is limited. We present both quantitative and qualitative analyses to evaluate the effec- tiveness and limitations of our approach1. 1 Introduction Recent task-oriented dialog systems (ToD) have achieved great success in intelligently communicat- ing with humans in natural languages (Chen et al., 2017; Bohus and Rudnicky, 2009). They are de- signed to fully assist users with widely heralded applications such as music playing, ticket ordering, or customer servicing (Zhang et al., 2020c). How- ever, most ToD systems are primarily established for English due to its ubiquity and the abundance of high-quality human annotations (Serban et al., 2015). Extending these services to global users may take tremendous efforts, especially in low- Labs.∗This work was done during an internship at Meta Reality 1Code available at https://github.com/waynewu6250/ zero-shot-multilingual-transfer-ACL-2023 resource languages where the collection of training corpus is labor-intensive. On the other hand, given a sizeable English di- alog corpus with standard dialog features shared across other languages, it is possible to transfer the knowledge and logic between languages via machine translation or cross-task alignments. Data- driven approaches (Schuster et al., 2019; Xiang et al., 2021) perform standard supervised training with translated dialogs, known as Translate-Train. Different pseudo-data pairs could be leveraged to enhance the multilingual model’s robustness. Nev- ertheless, a fine-grained machine translation system may not exist in an extremely low-resource lan- guage. The translation errors of entities in ground truth annotations (e.g. Indian could be translated in Chinese either to an adjective of Indian or In- dian people in different contexts.) can drastically influence how model is supervised. This primar- ily happens in dialog tasks like dialog state track- ing (DST) or natural language response generation (NLG) with language-sensitive outputs. Another line of approaches instead investigates cross-lingual transfer directly in …

Content truncated to 500 words