Nepali Speech Recognition Using CNN, GRU and CTC

Nepali Speech Recognition Using CNN, GRU and CTC Bharat Bhatta brb4344@gmail.com Basanta Joshi basanta@ioe.edu.np Ram Krishna Maharjhan rkmahajn@ioe.edu.np Department of Electronics and Computer Engineering Pulchowk Campus, Institute of Engineering Tribhuvan University Nepal Abstract Communication is an important part of life. To use communication technology efficiently we need to know how to use them or how to instruct these devices to perform tasks. Automatic speech recognition plays an important role in interaction with the technology. Nepali speech recognition involves in conversion of Nepali speech to its correct Nepali transcriptions. The purposed model consists of CNN, GRU and CTC network. The feature in the raw audio is extracted by using MFCC algorithm. CNN is for learning high level features. GRU is responsible for constructing the acoustic model. CTC is responsible for decoding. The dataset consists of 18 female speakers. It is provided by Open Speech and Language Resources. The build model can predict the with the WER of 11%. Keywords: Nepali Speech Recognition, Automatic Speech Recognition, Gated Recurrent Unit (GRU), Convolution Neural Network (CNN) The 32nd Conference on Computational Linguistics and Speech Processing (ROCLING 2020) Taipei, Taiwan, September 24–26, 2020. The Association for Computational Linguistics and Chinese Language Processing 238 1 Introduction Speaking and writing are the two important things that help us to communicate among us. Deficient of either writing or speaking affects our daily activates. Most of the people in rular area are able to speak properly but not able to write properly. Most of communication technology (gadgets, mobiles, computers etc) needs text as an input for their operation. To make familiar with the technology Automatic Speech Recognition (ASR) can play significant role. The Nepali ASR converts the spoken Nepali voice to its textual representation. The ASR can built by two different approaches. The first approach is traditional based that implement Hidden Markov Model (HMM) and Gaussian Mixture Model (GMM). The input feature vector is efficiently processed by the GMM and calculated the emission proba- bilities for each HMM states[1]. The second is to deep learning approach to build the acoustic model. The use of deep learning approach significantly increases the performance of the ASR system[2]. Traditional ASR system needs separate block of phonetic/linguistic constructs, acoustic model and the language model. The requirement of separate phonetic/linguistic constructs is eliminated by the deep learning approach[3]. Previously, a work is carried out to recognised the ten Nepali unique words[4] and another work is carried out taking the dataset of sports news. It consists of 1320 words among which 617 are unique[5]. The second model could not recognised the characters '�', '�व', 'के ', '�ल' that occurs very close together suggesting that the network learned them as single sounds although they are multiple characters. The CNN is used to learn the features that can easily distinguish those close words. This paper presents an idea to build the Nepali ASR system that can convert spoken Nepali language to its textual representation. The model used MFCC as input feature vector. These MFCC features area used …

Content truncated to 500 words