Lexicon-Free Conversational Speech Recognition with Neural Networks

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 345–354, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Lexicon-Free Conversational Speech Recognition with Neural Networks Andrew L. Maas∗, Ziang Xie∗, Dan Jurafsky, Andrew Y. Ng Stanford University Stanford, CA 94305, USA {amaas, zxie, ang}@cs.stanford.edu, jurafsky@stanford.edu Abstract We present an approach to speech recogni- tion that uses only a neural network to map acoustic input to characters, a character-level language model, and a beam search decoding procedure. This approach eliminates much of the complex infrastructure of modern speech recognition systems, making it possible to di- rectly train a speech recognizer using errors generated by spoken language understand- ing tasks. The system naturally handles out of vocabulary words and spoken word frag- ments. We demonstrate our approach us- ing the challenging Switchboard telephone conversation transcription task, achieving a word error rate competitive with existing base- line systems. To our knowledge, this is the first entirely neural-network-based system to achieve strong speech transcription results on a conversational speech task. We analyze qualitative differences between transcriptions produced by our lexicon-free approach and transcriptions produced by a standard speech recognition system. Finally, we evaluate the impact of large context neural network charac- ter language models as compared to standard n-gram models within our framework. 1 Introduction Users increasingly interact with natural language understanding systems via conversational speech in- terfaces. Google Now, Microsoft Cortana, and Apple Siri are all systems which rely on spoken language understanding (SLU), where transcribing ∗Authors contributed equally. speech is a single step within a larger system. Build- ing such systems is difficult because spontaneous, conversational speech naturally contains repetitions, disfluencies, partial words, and out of vocabulary (OOV) words (De Mori et al., 2008; Huang et al., 2001). Moreover, SLU systems must be robust to transcription errors, which can be quite high depend- ing on the task and domain. Modern systems for large vocabulary continuous speech recognition (LVCSR) use hidden Markov models (HMMs) to handle sequence processing, word-level language models, and a pronunciation lexicon to map words into phonetic pronunciations (Saon and Chien, 2012). Traditional systems use Gaussian mixture models (GMMs) to build a map- ping from sub-phonetic states to audio input fea- tures. The resulting speech recognition system con- tains many sub-components, linguistic assumptions, and typically over ten thousand lines of source code. Within the past few years LVCSR systems improved by replacing GMMs with deep neural networks (DNNs) (Dahl et al., 2011; Hinton et al., 2012), drawing on early work on with hybrid GMM-NN architectures (Bourlard and Morgan, 1993). Both HMM-GMM and HMM-DNN systems remain dif- ficult to build, and nearly impossible to efficiently optimize for downstream SLU tasks. As a result, SLU researchers typically operate on an n-best list of possible transcriptions and treat the LVCSR sys- tem as a black box. Recently Graves and Jaitly (2014) demonstrated an approach to LVCSR using a neural network trained with the connectionist temporal classifica- tion (CTC) loss function (Graves …

Content truncated to 500 words