Enhancing Automatic Speech Recognition Performance Through Multi-Speaker Text-to-Speech

Enhancing Automatic Speech Recognition Performance Through Multi-Speaker Text-to-Speech Po-Kai Chen Department of CSIE, National Central University pokaichen@g.ncu.edu.tw Hsin-Min Wang Institute of Information Science, Academia Sinica whm@iis.sinica.edu.tw Bing-Jhih Huang Department of CSIE, National Central University c72599@gmail.com Chi-Tao Chen Department of CSIE, National Central University apoman123@gmail.com Jia-Ching Wang Department of CSIE, National Central University jcw@csie.ncu.edu.tw Abstract In this study, we present a novel ap- proach to enhancing the performance of our Hakka Automatic Speech Recognition (ASR) model through the strategic use of Text-to-Speech (TTS) amplification tech- niques. Our investigation explores the inte- gration of diverse speakers to expand our training dataset, leading to a notable re- duction of Character Error Rate (CER) ap- proximately 0.2 in the validation set and approximately 3.96 on the test set. These compelling results affirm the effectiveness of multi-speaker TTS strategies in gener- ating ASR data, ultimately bolstering the resilience and precision of our ASR system. Keywords: Automatic Speech Recogni- tion, Multi-Speaker Text-to-Speech, Data ex- tension 1 Introduction In the ever-evolving landscape of Automatic Speech Recognition (ASR)(Mishaim Malik, 2020), the quest to unlock the potential of underrepresented languages and dialects re- mains a paramount challenge. Among these linguistic treasures, Hakka, a prominent lan- guage in Taiwan, has long awaited its mo- ment in the spotlight. This paper embarks on a transformative journey, harnessing state- of-the-art ASR models—Whisper and WavLM —as our guiding lights, and employing innova- tive strategies to bridge the gap between tech- nology and linguistic diversity. The founda- tion of our research rests on a meticulous cu- ration of training and testing data provided by the competition organizers, augmented by additional data sourced through diligent ef- forts. To further enhance our dataset, we turn to Text-to-Speech (TTS) systems—a powerful tool that aligns seamlessly with our vision of data augmentation for Hakka ASR. Through these efforts, we amplify our dataset’s rich- ness and diversity, fostering the robustness of our ASR system in the face of linguistic varia- tions. This paper unfolds as a testament to the resilience of underrepresented languages and the transformative power of cutting-edge ASR technologies. By integrating Whisper and WavLM, augmenting our data, and em- bracing TTS as an augmentation tool, we aim to amplify the voices of Hakka speakers,and fostering inclusivity. In the following sections, we delve into the intricacies of our methodol- ogy, present empirical findings, and engage in a nuanced discussion, all in the pursuit of ad- vancing Hakka ASR technology and transcend- ing the linguistic frontiers that beckon us for- ward. 2 Related Works 2.1 Whisper In the realm of ASR, the Whisper model (Rad- ford et al., 2023) stands as a beacon of in- novation and performance, symbolizing a cru- cial advancement in the pursuit of effective and versatile ASR solutions. As we delve into the intricacies of spoken language under- standing, Whisper emerges as a transforma- tive force, exemplifying excellence in multi- lingual ASR systems. A defining feature of Whisper is its adaptability, rendering it profi- cient in both major languages and underrepre- The 35th Conference on …

Content truncated to 500 words