The NTNU Taiwanese ASR System for Formosa Speech Recognition Challenge 2020

Computational Linguistics and Chinese Language Processing Vol. 26, No. 1, June 2021, pp. 1-16 1  The Association for Computational Linguistics and Chinese Language Processing The NTNU Taiwanese ASR System for Formosa Speech Recognition Challenge 2020 Fu-An Chao, Tien-Hong Lo, Shi-Yan Weng, Shih-Hsuan Chiu, Yao-Ting Sung, and Berlin Chen Abstract This paper describes the NTNU ASR system participating in the Formosa Speech Recognition Challenge 2020 (FSR-2020) supported by the Formosa Speech in the Wild project (FSW). FSR-2020 aims at fostering the development of Taiwanese speech recognition. Apart from the issues on tonal and dialectical variations of the Taiwanese language, speech artificially contaminated with different types of real-world noise also has to be dealt with in the final test stage; all of these make FSR-2020 much more challenging than before. To work around the under-resourced issue, the main technical aspects of our ASR system include various deep learning techniques, such as transfer learning, semi-supervised learning, front-end speech enhancement and model ensemble, as well as data cleansing and data augmentation conducted on the training data. With the best configuration, our system obtains 13.1 % syllable error rate (SER) on the final-test set, achieving the first place among all participating systems on Track 3. Keywords: Formosa Speech Recognition Challenge, Deep Learning, Transfer Learning, Semi-supervised Training 1. Introduction Due to the rapid developments of deep learning, deep neural network (DNN) based techniques have enjoyed widespread adoption in the automatic speech recognition (ASR) community.  Department of Computer Science and Information Engineering, National Taiwan Normal University, Taipei, Taiwan E-mail: {fuann, tienhonglo, 60947007S, shchiu, sungtc, berlin}@ntnu.edu.tw  Department of Educational Psychology and Counseling, National Taiwan Normal University, Taipei, Taiwan  Institute for Research Excellence in Learning Sciences, National Taiwan Normal University, Taipei, Taiwan 2 Fu-An Chao et al. ASR has also found its applications in many different areas, ranging from interactive voice response (IVR) services and personal assistants, for which people can interact with the machine naturally by using their own voices, meeting transcription, speech translation to speech summarization. Nowadays, some top-of-the-line ASR systems can even reach the performance level of professional human annotators in English, a dominant language in the world. However, in real-world scenarios, there exist some languages that are resource-poor or even endangered. For example, although both Mandarin (a.k.a. Pǔtōnghuà and Huáyǔ) and Taiwanese (a.k.a. Taiwanese Hokkien, Hoklo, Taigi, Southern Min and Min-Nan) are Chinese dialects spoken by large populations of people, the latter is underexplored and has far less ASR training data made publicly available than the former, which causes the performance of Taiwanese ASR systems to fall short of expectations. Furthermore, distinct from Mandarin, Taiwanese has a wide variety of pronunciation traits that can be attributed to the influences from disparate languages like Formosan, Dutch, Japanese, and among others (“Taiwanese Hokkien,” 2021). Despite there are still 70% of the population in Taiwan who use Taiwanese to communicate, most of the people, young generations in particular, have only limited vocabulary and cannot speak Taiwanese fluently. Therefore, in addition to the continued promotion …

Content truncated to 500 words