A Survey of Multilingual Models for Automatic Speech Recognition

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 5071–5079 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 5071 A Survey of Multilingual Models for Automatic Speech Recognition Hemant Yadav, Sunayana Sitaram Microsoft Research India Bangalore t-hyadav@microsoft.com, sunayana.sitaram@microsoft.com Abstract Although Automatic Speech Recognition (ASR) systems have achieved human-like performance for a few languages, the majority of the world’s languages do not have usable systems due to the lack of large speech datasets to train these models. Cross-lingual transfer is an attractive solution to this problem, because low-resource languages can potentially benefit from higher-resource languages either through transfer learning, or being jointly trained in the same multilingual model. The problem of cross-lingual transfer has been well studied in ASR, however, recent advances in Self Supervised Learning are opening up avenues for unlabeled speech data to be used in multilingual ASR models, which can pave the way for improved performance on low-resource languages. In this paper, we survey the state of the art in multilingual ASR models that are built with cross-lingual transfer in mind. We present best practices for building multilingual models from research across diverse languages and techniques, discuss open questions and provide recommendations for future work. Keywords: speech recognition, multilingual, low-resource languages 1. Introduction The field of Automatic Speech Recognition (ASR) has made significant progress over the last few years due to the advent of Deep Neural Network (DNN) based mod- els. Most notably, on benchmarks such as Switchboard, models have now achieved human parity (Xiong et al., 2016), which means that models are able to transcribe human speech with the same or better accuracy as hu- mans. However, due to the large data requirement of DNN-based models, most of these advances have been restricted to a few languages of the world, which have large datasets of transcribed speech. Languages that do not have much transcribed and/or untranscribed speech, that is, low-resource languages have been left behind. (Joshi et al., 2020) classify languages based on re- sources for Natural Language Processing (NLP) and find that almost 90% of the world’s population speaks languages that are very low resource and are thus not benefited by language technologies. Since speech datasets are usually more expensive to build than text datasets, the situation may be even more stark for ASR. This is a lost opportunity for serving language commu- nities where speech is the only medium of expression, owing to low literacy or the lack of a standardized writ- ing system. The problem of ASR for low resource languages has been well studied starting from traditional HMM- GMM based models, to the transformer-based models used today. The main technique used for addressing the lack of resources in the target language is cross-lingual transfer, in which models or resources from a higher re- source language are adapted to the target low resource language using adaptation data. Cross-lingual transfer may take place using models trained on a single high resource language, or multiple languages with …

Content truncated to 500 words