Acoustic-to-Word Models with Conversational Context Information

Proceedings of NAACL-HLT 2019, pages 2766–2771 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2766 Acoustic-to-Word Models with Conversational Context Information Suyoun Kim1 and Florian Metze2 1Electrical & Computer Engineering 2Language Technologies Institute, School of Computer Science Carnegie Mellon University {suyoung1, fmetze}@andrew.cmu.edu Abstract Conversational context information, higher- level knowledge that spans across sentences, can help to recognize a long conversation. However, existing speech recognition models are typically built at a sentence level, and thus it may not capture important conversational context information. The recent progress in end-to-end speech recognition enables inte- grating context with other available informa- tion (e.g., acoustic, linguistic resources) and directly recognizing words from speech. In this work, we present a direct acoustic-to- word, end-to-end speech recognition model capable of utilizing the conversational context to better process long conversations. We eval- uate our proposed approach on the Switch- board conversational speech corpus and show that our system outperforms a standard end-to- end speech recognition system. 1 Introduction Many real-world speech recognition applications, including teleconferencing, and AI assistants, re- quire recognizing and understand long conversa- tions. In a long conversation, there exists the ten- dency of semantically related words or phrases re- occur across sentences, or there exists topical co- herence. Thus, such conversational context infor- mation, higher-level knowledge that spans across sentences, provides important information that can improve speech recognition. However, the long conversations typically split into short sentence- level audios to make building speech recognition models computationally feasible in current state- of-the-art recognition systems (Xiong et al., 2017; Saon et al., 2017). Over the years, there have been many studies have attempted to inject a longer context infor- mation into language models. Based on a re- current neural network (RNNs) language models (Mikolov et al., 2010), (Mikolov and Zweig, 2012; Wang and Cho, 2015; Ji et al., 2015; Liu and Lane, 2017; Xiong et al., 2018), proposed using a con- text vector that would encode the longer context information as an additional network input. How- ever, all of these models have been developed on text data, and therefore, it must still be integrated with a conventional acoustic model which is built separately without a longer context information, for speech recognition on long conversations. Recently, new approaches to speech recogni- tion models integrate all available information (e.g. acoustic, linguistic resources) in a so-called end-to-end manner proposed in (Graves et al., 2006; Graves and Jaitly, 2014; Hannun et al., 2014; Miao et al., 2015; Bahdanau et al., 2014; Chorowski et al., 2014, 2015; Chan et al., 2015; Kim et al., 2017). In these approaches, a single neural network is trained to recognize graphemes or even words from speech directly. Especially, the model using semantically meaningful units, such as words or sub-word (Sennrich et al., 2015), rather than graphemes have been showing promis- ing results (Audhkhasi et al., 2017b; Li et al., 2018; Soltau et al., 2016; Zenkel et al., 2017; Palaskar and Metze, 2018; Sanabria and Metze, 2018; Rao et al., 2017; Zeyer …

Content truncated to 500 words