Meta-Transfer Learning for Code-Switched Speech Recognition

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3770–3776 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3770 Meta-Transfer Learning for Code-Switched Speech Recognition Genta Indra Winata†∗, Samuel Cahyawijaya∗, Zhaojiang Lin, Zihan Liu, Peng Xu, Pascale Fung Center for Artificial Intelligence Research (CAiRE) Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology †giwinata@connect.ust.hk Abstract An increasing number of people in the world today speak a mixed-language as a result of being multilingual. However, building a speech recognition system for code-switching remains difficult due to the availability of lim- ited resources and the expense and significant effort required to collect mixed-language data. We therefore propose a new learning method, meta-transfer learning, to transfer learn on a code-switched speech recognition system in a low-resource setting by judiciously extract- ing information from high-resource monolin- gual datasets. Our model learns to recog- nize individual languages, and transfer them so as to better recognize mixed-language speech by conditioning the optimization on the code- switching data. Based on experimental re- sults, our model outperforms existing base- lines on speech recognition and language mod- eling tasks, and is faster to converge. 1 Introduction In bilingual or multilingual communities, speak- ers can easily switch between different languages within a conversation (Wang et al., 2009). Peo- ple who know how to code-switch will mix lan- guages in response to social factors as a way of communicating in a multicultural society. Gen- erally, code-switching speakers switch languages by taking words or phrases from the embedded language to the matrix language. This can oc- cur within a sentence, which is known as intra- sentential code-switching or between two matrix language sentences, which is called inter-sentential code-switching (Heredia and Altarriba, 2001). Learning a code-switching automatic speech recognition (ASR) model has been a challenging task for decades due to data scarcity and diffi- culty in capturing similar phonemes in different ∗These two authors contributed equally. ∇{� � ∇�ℎ p{� py� p�ℎ p{� py� p�ℎ � w) x) ∇y� Figure 1: Illustration of (a) joint training and (b) meta- transfer learning. The solid lines show the optimiza- tion path. The orange circles represent the monolin- gual source language, and the white circles represent the code-switching target language. The lower black circle in (b) is closer to Tcs than that in (a). languages. Several approaches have focused on generating synthetic speech data from monolingual resources (Nakayama et al., 2018; Winata et al., 2019). However, these methods are not guaran- teed to generate natural code-switching speech or text. Another line of work explores the feasibil- ity of leveraging large monolingual speech data in the pre-training and applying fine-tuning on the model using a limited source of code-switching data, which has been found useful to improve the performance (Li et al., 2011; Winata et al., 2019). However, the transferability of these pretraining approaches is not optimized on extracting useful knowledge from each individual languages in the context of code-switching, and even after the fine- tuning …

Content truncated to 500 words