Progress in Multilingual Speech Recognition for Low Resource Languages Kurmanji Kurdish, Cree and Inuktut

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6420–6428 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6420 Progress in Multilingual Speech Recognition for Low Resource Languages Kurmanji Kurdish, Cree and Inuktut Vishwa Gupta and Gilles Boulianne Centre de Recherche Informatique de Montr´eal (CRIM), Quebec, Canada {vishwa.gupta,gilles.boulianne}@crim.ca Abstract This contribution presents our efforts to develop the automatic speech recognition (ASR) systems for three low resource languages: Kurmanji Kurdish, Cree and Inuktut. As a first step, we generate multilingual models from acoustic training data from 12 different languages in the hybrid DNN/HMM framework. We explore different strategies for combining the phones from different languages: either keep the phone labels separate for each language or merge the common phones. For Kurmanji Kurdish and Inuktut, keeping the phones separate gives much lower word error rate (WER), while merging phones gives lower WER for Cree. These WER are lower than training the acoustic models separately for each language. We also compare two different DNN architectures: factored time delay neural network (TDNN-F), and bidirectional long short-term memory (BLSTM) acoustic models. The TDNN-F acoustic models give significantly lower WER for Kurmanji Kurdish and Cree, while BLSTM acoustic models give significantly lower WER for Inuktut. We also show that for each language, training multilingual acoustic models by one more epoch with acoustic data from that language reduces the WER significantly. We also added 512-dimensional embedding features from cross-lingual pre-trained wav2vec2.0 XLSR-53 models, but they lead to only a small reduction in WER. Keywords: multilingual acoustic models, OpenASR20 Challenge, factored TDNN, bidirectional LSTM, wav2vec em- bedding 1. Introduction The low resource transcription and keyword spotting effort received a great impetus from the IARPA Ba- bel program1. The main goal of the program was to improve the performance of keyword search on lan- guages with very little transcribed data (low-resource languages). Data from 26 languages was collected with certain languages being held out as surprise languages to test the ability of the teams to rapidly build a sys- tem for a new language2. Many different DNN train- ing algorithms have been experimented with within the Babel program (Gales et al., 2014) (Knill et al., 2014) (Huang et al., 2013) (Trmal et al., 2014) (Chen et al., 2013) (Zhang et al., 2014). In (Gales et al., 2014) they experiment with both DNN and tandem systems and achieve token error rates (TER) between 60% and 77% with limited language packs (10 hours of training audio), depending on the language and training algo- rithms. They also experiment with data augmentation by automatically labeling untranscribed data. In (Huang et al., 2013), the authors keep the senones separate for each language in the softmax layer, while sharing the hidden layers across language. In (Vu et al., 2014), the authors experiment with two different ways of generating multilingual phone sets: keep phones for each language separate, or merge phones that have the same IPA symbols across different languages. In their study, they show that keeping …

Content truncated to 500 words