Joint CTC/attention decoding for end-to-end speech recognition

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 518–529 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1048 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 518–529 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1048 Joint CTC/attention decoding for end-to-end speech recognition Takaaki Hori, Shinji Watanabe, John R. Hershey Mitsubishi Electric Research Laboratories (MERL) {thori,watanabe,hershey}@merl.com Abstract End-to-end automatic speech recognition (ASR) has become a popular alterna- tive to conventional DNN/HMM sys- tems because it avoids the need for linguistic resources such as pronuncia- tion dictionary, tokenization, and context- dependency trees, leading to a greatly simplified model-building process. There are two major types of end-to-end archi- tectures for ASR: attention-based meth- ods use an attention mechanism to per- form alignment between acoustic frames and recognized symbols, and connection- ist temporal classification (CTC), uses Markov assumptions to efficiently solve sequential problems by dynamic program- ming. This paper proposes a joint de- coding algorithm for end-to-end ASR with a hybrid CTC/attention architecture, which effectively utilizes both advantages in decoding. We have applied the pro- posed method to two ASR benchmarks (spontaneous Japanese and Mandarin Chi- nese), and showing the comparable per- formance to conventional state-of-the-art DNN/HMM ASR systems without lin- guistic resources. 1 Introduction Automatic speech recognition (ASR) is currently a mature set of technologies that have been widely deployed, resulting in great success in interface applications such as voice search. A typical ASR system is factorized into several modules includ- ing acoustic, lexicon, and language models based on a probabilistic noisy channel model (Jelinek, 1976). Over the last decade, dramatic improve- ments in acoustic and language models have been driven by machine learning techniques known as deep learning (Hinton et al., 2012). However, current systems lean heavily on the scaffolding of complicated legacy architectures that grew up around traditional techniques. For example, when we build an acoustic model from scratch, we have to first build hidden Markov model (HMM) and Gaussian mixture model (GMM) followed by deep neural networks (DNN). In addition, the factorization of acoustic, lexicon, and language models is derived by conditional in- dependence assumptions (especially Markov as- sumptions), although the data do not necessarily follow such assumptions leading to model mis- specification. This factorization form also yields a local optimum since the above modules are optimized separately. Further, to well factorize acoustic and language models, the system requires linguistic knowledge based on a lexicon model, which is usually based on a hand-crafted pronun- ciation dictionary to map word to phoneme se- quence. In addition to the pronunciation dictio- nary issue, some languages, which do not ex- plicitly have a word boundary, need language- specific tokenization modules (Kudo et al., 2004; Bird, 2006) for language modeling. Finally, in- ference/decoding has to be performed by integrat- ing all modules resulting in complex decoding. Consequently, it is quite difficult for non-experts to use/develop ASR systems for new applications, especially for new …

Content truncated to 500 words