Evaluation of Feature-Space Speaker Adaptation for End-to-End Acoustic Models

Evaluation of Feature-Space Speaker Adaptation for End-to-End Acoustic Models Natalia Tomashenko1,2 and Yannick Est`eve1 1 LIUM, University of Le Mans, France 2 ITMO University, Saint-Petersburg, Russia {natalia.tomashenko, yannick.esteve}@univ-lemans.fr Abstract This paper investigates speaker adaptation techniques for bidirectional long short term memory (BLSTM) recurrent neural network based acoustic models (AMs) trained with the connectionist temporal classification (CTC) objective function. BLSTM-CTC AMs play an important role in end-to-end automatic speech recognition systems. However, there is a lack of research in speaker adaptation algorithms for these models. We explore three different feature-space adaptation approaches for CTC AMs: feature-space maximum linear regression, i-vector based adaptation, and maximum a posteriori adaptation using GMM-derived features. Experimental results on the TED-LIUM corpus demonstrate that speaker adaptation, applied in combination with data augmentation techniques, provides, in an unsupervised adaptation mode, for different test sets, up to 11–20% of relative word error rate reduction over the baseline model built on the raw filter-bank features. In addition, the adaptation behavior is compared for BLSTM-CTC AMs and time-delay neural network AMs trained with the cross-entropy criterion. Keywords: Speaker adaptation, end-to-end speech recognition, GMM-derived features, deep neural network, acoustic model 1. Introduction Recently, various neural end-to-end approaches to auto- matic speech recognition (ASR) have been proposed in the literature (Hannun et al., 2014; Bahdanau et al., 2016; Col- lobert et al., 2016; Fritz and Burshtein, 2017; Chan et al., 2016; Audhkhasi et al., 2017). End-to-end acoustic mod- els (AMs) (Chorowski et al., 2014; Graves and Jaitly, 2014; Miao et al., 2015; Zhang et al., 2017) attempt to map an acoustic signal to a phoneme or grapheme sequence directly by means of neural network models. They have been de- veloped as an alternative to the traditional hybrid approach based on hidden Markov models coupled to deep neural networks (HMM-DNNs) (Hinton et al., 2012). Speaker adaptation is an essential component of state-of- the-art hybrid HMM-DNN AMs, and a variety of adapta- tion methods have been developed for DNNs. They include linear transformations, that can be applied at different lev- els of the DNN-HMM architecture (Gemello et al., 2006; Seide et al., 2011); regularization techniques, such as L2- prior regularization (Liao, 2013) or Kullback-Leibler diver- gence regularization (Yu et al., 2013); model-space adap- tation (Siniscalchi et al., 2013; Swietojanski and Renals, 2014; Huang et al., 2014); multi-task learning (MTL) (Price et al., 2014; Li et al., 2015; Huang et al., 2015); factor- ized adaptation (Li et al., 2014); adaptation with speaker codes (Xue et al., 2014); the use of auxiliary features, such as i-vectors (Saon et al., 2013; Senior and Lopez-Moreno, 2014) or GMM-derived (GMMD) features (Tomashenko and Khokhlov, 2014), and many others. However, the major part of the published works, devoted to end-to-end technology, does not use any speaker adaptation techniques. This lack may be justified by the strong focus of these papers on the neural core of the technology they introduce. A few papers have offered some preliminary and promising information about the benefits provided by some speaker adaptation techniques to end-to-end AMs. …

Content truncated to 500 words