The Taiwan AI Labs Hakka ASR System for Formosa Speech Recognition Challenge 2023

The Taiwan AI Labs Hakka ASR System for Formosa Speech Recognition Challenge 2023 Yuan-Hsiang Lu Taiwan AI Labs b07901030@ntu.edu.tw Chung-Yi Li Taiwan AI Labs chungyi.li@ailabs.tw Zih-Wei Lin Taiwan AI Labs zweilin@ailabs.tw Abstract This paper introduces the architecture and training approach of the Taiwan AI Labs Hakka automatic speech recognition (ASR) system for the Formosa Speech Recognition Challenge 2023 (FSR2023). Overall, this Hakka ASR system consists of an acoustic model trained based on the HuBERT (Hsu et al., 2021) model and a 5-gram language model. The HuBERT acoustic model is built upon the open-source sequence modeling toolkit, fairseq (Ott et al., 2019), while the language model is implemented using KenLM (Heafield, 2011). In the end, this system achieved a CER of 8.28% and a WER of 3.14% on track1 (character) and track2 (pinyin) of the pilot test dataset for the FSR2023, respectively. In the FSR2023 final dataset, the system achieved a CER of 30.85% (reading: 10.85%, spon- taneous: 33.94%) and a WER of 17.93% (reading: 6.16%, spontaneous: 19.66%) on track1 (character) and track2 (pinyin), re- spectively. Keywords: FSR2023, Hakka, Automatic Speech Recognition, HuBERT, KenLM 1 Introduction In recent times, there have been significant ad- vancements in the field of automatic speech recognition (ASR). Self-supervised learning- based models such as HuBERT, and large- scale weakly supervised Seq2seq models such as Whisper (Radford et al., 2023) have emerged one after another, and they have shown excellent performance in ASR tasks, ei- ther in English or Chinese. Even in the case of Taiwanese (Hokkien), one of the native lan- guages of Taiwan, as it gradually gains more attention, numerous new datasets have been collected, and many individuals have invested in model training, resulting in substantial im- provements. However, Hakka, another native language of Taiwan, has not received as much attention. Fortunately, with the establishment and col- lection of the HAT-Vol1 dataset and the Tai- wan Hakka Corpus (Hakka Affairs Council, 2022) dataset, along with the organization of the FSR2023-Hakka ASR competition, many researchers and research teams have ventured into the field of Hakka ASR, leading to no- table progress. With the help of these Hakka datasets, the Taiwan AI Labs Hakka ASR sys- tem has also made a significant breakthrough in this area. The Taiwan AI Labs Hakka ASR system is a neural network-based ASR model com- posed of two main components: an acoustic model and a language model. For the acoustic model, we have adopted the HuBERT model as our architecture. The HuBERT model is a self-supervised audio representation learning model designed to learn audio feature repre- sentations from unlabeled audio data. It is tailored for audio processing tasks, using a pretraining strategy similar to the BERT (De- vlin et al., 2018) model but specialized for the audio domain. After pretraining, the model needs to undergo fine-tuning with labeled data to adapt to Hakka ASR tasks. In the pretraining stage of our acoustic model, the unlabeled training data for pre- training included HAT-Vol1, Taiwan Hakka Corpus, Hakka YouTube videos’ audio data, and Hakka …

Content truncated to 500 words