Syllable based DNN-HMM Cantonese Speech to Text System

Syllable based DNN-HMM Cantonese Speech-to-Text System Timothy WONG, Claire W. Y. LI, Sam LAM, Billy CHIU, Qin LU, Minglei LI, Dan XIONG, Roy S. YU, Vincent T.Y. NG Department of Computing, The Hong Kong Polytechnic University, Hong Kong timtim.hk@connect.polyu.hk, {cswyli, cswolam, bchiu, csluqin, csmli,csdxiong, cssyu, cstyng}@comp.polyu.edu.hk Abstract This paper reports our work on building up a Cantonese Speech-to-Text (STT) system with a syllable based acoustic model. This is a part of an effort in building a STT system to aid dyslexic students who have cognitive deficiency in writing skills but have no problem expressing their ideas through speech. For Cantonese speech recognition, the basic unit of acoustic models can either be the conventional Initial-Final (IF) syllables, or the Onset-Nucleus-Coda (ONC) syllables where finals are further split into nucleus and coda to reflect the intra-syllable variations in Cantonese. By using the Kaldi toolkit, our system is trained using the stochastic gradient descent optimization model with the aid of GPUs for the hybrid Deep Neural Network and Hidden Markov Model (DNN-HMM) with and without I-vector based speaker adaptive training technique. The input features of the same Gaussian Mixture Model with speaker adaptive training (GMM-SAT) to DNN are used in all cases. Experiments show that the ONC-based syllable acoustic modeling with I-vector based DNN-HMM achieves the best performance with the word error rate (WER) of 9.66% and the real time factor (RTF) of 1.38812. Keywords: Cantonese Speech Recognition, DNN-HMM, Onset-Nucleus-Coda syllable scheme, Kaldi Toolkit. 1. Introduction In hybrid context dependent DNN-HMM speech recognition methods, an artificial neural network (ANN) with multiple non-linear hidden layers is trained to output posterior probabilities of output frame labels corresponding to tied HMM triphone states (senones). The input of a higher-dimensional feature vector is composed from consecutive concatenated frames in which Mel-Frequency Cepstral Coefficients (MFCC) or filter-bank features are successively normalized and transformed on a per speaker basis. While the alignments and output labels for DNN training come from the GMM-HMM system with a set of HMM triphone states and their corresponding Gaussian models. Due to power of DNN and its efficiency in learning discriminative features, recent attempts have used much higher dimensional input features to DNN-HMM systems such as the I-vector based speaker adaptive training (SAT) techniques, in which several hundred dimensional per-speaker I-vector features as extra input are added to the conventional 40 dimensional GMM-SAT features as the final input to the neural net (Karafiátet al., 2011; Gupta et al,, 2014). The GMM-SAT features are spliced across several to tens of consecutive frames rather than just one frame in a GMM system. Related works have been done using syllable-based acoustic modeling on large-vocabulary continuous speech recognition (LVCSR) for both monosyllabic and polysyllabic languages, including Mandarin (Lee et al., 1993; Pan et al., 2012; Deng Li and Li Xiao, 2013; Li et al., 2013; Hu et al., 2014; X. Li, and X. Wu, 2014) and West languages (Hinton et al., 2012; A. Mohamed, 2012; Swietojanski et al., 2013; Gupta & Boulianne, 2013; Schmidhuber 2015). However, automatic STT …

Content truncated to 500 words