Taiwanese Speech Recognition Based on Hybrid Deep Neural Network Architecture

Taiwanese Speech Recognition Based on Hybrid Deep Neural Network Architecture Yu-Fu Yeh , Bo-Hao Su , Yang-Yen Ou , Jhing-Fa Wang Department of Electrical Engineering National Cheng Kung University Tainan, Taiwan n26070106@gs.ncku.edu.tw , xtlettle99360017@gmail.com , ouyang0916@gmail.com , wangjf@mail.ncku.edu.tw An-Chao Tsai Department of Computer Science and Entertainment Technology Tajen University, Pingtung, Taiwan actsai@tajen.edu.tw Abstract In this research, we developed the Taiwanese speech recognition system which used the Kaldi toolkit to implement. The Taiwanese corpus was collected by Taiwan Taiwanese National Reading Competition and Classmate Recording, and a total of about 11 hours of audio files were collected. Because the training data is small dataset, two audio augmentation methods are used to increase the training data, so that the acoustic model can be more robust and more effective training. One method is speed perturbation, which speeds up the original data by 1.1 times and slows it down by 0.9 times. Another method is to use multi-condition training data to simulate reverberation of the original speech and add background noise. The background noise includes music, speech, and noise. The acoustic model is trained for different hybrid deep neural network architectures which can use the advantages of each neural network by hybrid different neural networks, including TDNN, CNN-TDNN and CNN-LSTM-TDNN. In the experimental results, the CER in the domain of language modeling reaches 3.95%, and the CER of online decoding test is 3.06%. Compared with other researches on Taiwanese speech recognition of similar dataset size, the recognition results are better than other studies. Keywords: Speech Recognition, Taiwanese, Data Augmentation, Deep Neural Network Acoustic Model. The 32nd Conference on Computational Linguistics and Speech Processing (ROCLING 2020) Taipei, Taiwan, September 24–26, 2020. The Association for Computational Linguistics and Chinese Language Processing 102 1. Introduction In the past few years, more and more products using speech recognition technology. Because these speech recognition applications make people's lives more and more convenient, no longer need to type to allow the machine to receive our message input. Taiwanese language is one of the commonly used languages of Taiwanese. From [1], we can know that in 2013, the social change survey results showed that 31.4% of the people in the family spoke Mandarin Chinese most often, and 44.2% of them spoke Taiwanese most often. 19.5% of the people advocate using both Mandarin and Taiwanese, but the proportion of the older generation is much larger than that of the younger generation, which shows that Taiwanese is still the main language for the elderly. Most of them are learning by word of mouth, which leads to relatively scarce resources in Taiwanese. It makes the research of Taiwanese-related technologies much more difficult, and also causes people who speak Taiwanese to not enjoy these conveniences. Therefore, we have established a Taiwanese dataset and a Taiwanese speech recognition system for this problem. 2. Related Work Establishes a deep neural network architecture in kaldi[2], the input features used in addition to the Mel frequency cepstral coefficients[3] will also concatenate the ivector feature [4], which is a feature …

Content truncated to 500 words