Multilingual Transfer Learning for Children Automatic Speech Recognition

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7314–7320 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7314 Multilingual Transfer Learning for Children Automatic Speech Recognition Thomas Rolland1,2 Alberto Abad1,2 Catia Cucchiarini3 Helmer Strik3 1INESC-ID, Lisbon, Portugal 2Instituto Superior T´ecnico, Universidade de Lisboa, Portugal 3Centre for Language and Speech Technology (CLST), Radboud University Nijmegen, The Netherlands Abstract Despite recent advances in automatic speech recognition (ASR), the recognition of children’s speech still remains a significant challenge. This is mainly due to the high acoustic variability and the limited amount of available training data. The latter problem is particularly evident in languages other than English, which are usually less-resourced. In the current paper, we address children ASR in a number of less-resourced languages by combining several small-sized children speech corpora from these languages. In particular, we address the following research question: Does a novel two-step training strategy in which multilingual learning is followed by language-specific transfer learning outperform conventional single language/task training for children speech, as well as multilingual and transfer learning alone? Based on previous experimental results with English, we hypothesize that multilingual learning provides a better generalization of the underlying characteristics of children’s speech. Our results provide a positive answer to our research question, by showing that using transfer learning on top of a multilingual model for an unseen language outperforms conventional single language-specific learning. Keywords: children speech, children speech recognition, ASR, multilingual training, transfer learning 1. Introduction Recently, significant improvements have been achieved by Automatic Speech Recognition (ASR) systems through the application of deep learning approaches. However, children ASR still represents a significant challenge, as testified by the performance drop of the current state-of-the-art systems compared to adult speech. This degradation can be partially attributed to the high acoustic variability in children speech caused by de- velopmental changes of the speech production appara- tus (Gerosa et al., 2009; Wilpon and Jacobsen, 1996). Such physical changes lead to different formant and fundamental frequency locations (Lee et al., 1999). Moreover, limited linguistic and phonetic knowledge of children can also contribute to performance degra- dation (Wilpon and Jacobsen, 1996). Finally, the per- formance gap between children and adult ASR can be explained by a data scarcity problem. Indeed, current speech recognition systems are based on deep learning, for which the amount of data used is essential. De- spite recent efforts to collect larger datasets of chil- dren speech (Ward et al., 2013), most of the pub- licly available children corpora contain less than fifty hours of speech, while adult speech corpora contain- ing hundreds (or even thousands) of hours can be eas- ily found (Panayotov et al., 2015). The problem of data scarcity is particularly acute for languages other than English, for which, in general, fewer resources are available. This can be seen in (Liao et al., 2015), in which the authors used a large amount of children speech –comparable to an adult speech corpus– to train a convolutional long-short-term-memory deep neural …

Content truncated to 500 words