Accelerating Hakka Speech Recognition Research and Development Using the Whisper Model

Abstract Language preservation has become increasingly urgent with globalization and rapid technological advancement. Minority languages, such as Hakka, are particularly vulnerable. This study aims to expedite the research and development of Hakka speech recognition by integrating Open AI's Whisper model with online Hakka speech resources. This study developed an efficient speech recognition model for Hakka, a low-resource language, and provided insights into its applications for preserving and popularizing Hakka culture. This paper addresses the challenge of building an Automatic Speech Recognition (ASR) model for the Hakka language. Utilizing Open AI's Whisper technology, this study presents a complete workflow for training and deploying a Hakka language ASR model. The end product could be a vital tool for digital Hakka language education and intelligent living applications. Keywords: Automatic Speech Recognition, Hakka Speech Recognition, Whisper Model, Minority languages 1 Introduction The preservation and promotion of minority languages like Hakka are becoming more crucial. One way to promote a language's usage and preservation is through technology. This paper explores the potential of using Open AI's Whisper model to accelerate the development of a Hakka speech recognition system. By combining the Whisper model with online Hakka speech resources, this study seeks to create an effective, scalable speech recognition model for this low- resource language. For numerous languages globally, there is an insufficient amount of annotated speech data to train an Automatic Speech Recognition (ASR) model effectively (Scharenborg et al., 2017). One of the most considerable challenges in Hakka language AI technology is developing an efficient Hakka Language ASR model. Achieving this would advance digital education in the Hakka language substantially. Not only could such a system evaluate the pronunciation accuracy of Hakka words and sentences, but it could also serve as an aid in teaching Hakka pronunciation. This paper discusses constructing a Hakka language ASR model using Whisper, an ASR neural network model provided by OpenAI (Radford et al., 2023), and online resources such as the Hakka Language Database, YouTube videos, and online education platforms. 2 Database Profile The database employed in this research constitutes a diverse collection of sources, encompassing the Hakka Language Database, YouTube videos, online educational materials, multimedia news content, and electronic dictionaries. This eclectic assortment of resources was chosen to construct a robust and all-encompassing dataset, affording the model the opportunity to glean insights from various accents, pronunciations, and lexical contexts. For the training phase, this study made use of data contributed by the 2023 Formosa Speech Recognition Challenge (Hakka ASR), Accelerating Hakka Speech Recognition Research and Development Using the Whisper Model Ching-Yuan Chen Department of Cultural Creativity and Digital Marketing, National United University minicr1234@gmail.com Chen-Chi Chang* Department of Cultural Creativity and Digital Marketing, National United University kiwi@gm.nuu.edu.tw *Corresponding author Yun-Hsiang Hsu Department of Cultural Creativity and Digital Marketing, National United University yuripeyamashita@gmail.com The 35th Conference on Computational Linguistics and Speech Processing (ROCLING 2023) Taipei City, Taiwan, October 20-21, 2023. The Association for Computational Linguistics and Chinese Language Processing specifically the FSR-2023-Hakka-Lavalier-Train dataset, in conjunction with a Hakka speech dataset licensed …

Content truncated to 500 words