Audio Segmentation for Robust Real-Time Speech Recognition Based on Neural Networks

Audio Segmentation for Robust Real-Time Speech Recognition Based on Neural Networks Micha Wetzel, Matthias Sperber, Alexander Waibel Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology, Germany micha.wetzel@student.kit.edu, {matthias.sperber, waibel}@kit.edu Abstract Speech that contains multimedia content can pose a serious challenge for real-time automatic speech recognition (ASR) for two reasons: (1) The ASR produces meaningless output, hurting the readability of the transcript. (2) The search space of the ASR is blown up when multimedia content is encoun- tered, resulting in large delays that compromise real-time re- quirements. This paper introduces a segmenter that aims to remove these problems by detecting music and noise seg- ments in real-time and replacing them with silence. We pro- pose a two step approach, consisting of frame classification and smoothing. First, a classifier detects speech and mul- timedia on the frame level. In the second step the smooth- ing algorithm considers the temporal context to prevent rapid class fluctuations. We investigate in frame classification and smoothing settings to obtain an appealing accuracy-latency- tradeoff. The proposed segmenter yields increases the tran- script quality of an ASR system by removing on average 39 % of the errors caused by non-speech in the audio stream, while maintaining a real-time applicable delay of 270 mil- liseconds. 1. Introduction Automatic speech recognition (ASR) systems generally re- spond poorly to music and noise input. The recognition de- lay increases due to a missing likely interpretation of the au- dio blowing up the search space. The number of insertions increases in those segments due to falsely detected speech. Filtering those segments out is desirable to increase the ac- curacy of the ASR and to improve the average recognition speed. This is especially important for real-time systems, where a small delay is required, and processing power is gen- erally limited. According to our experience, a delay of less than 3 seconds is desirable in a real-time environment, al- though previous work has assumed up to 5 seconds [2]. The drawback in existing segmentation models for speech / non-speech classes is that they induce a high (> 1 s) latency and are not evaluated and optimized as a preprocess- ing step for ASR systems. Usually only the accuracy of clas- sifying single frames with some additional temporal context is investigated [1, 7, 5]. El-Maleh et al. reported an accu- racy of 96 % when classifying audio frames of size 1 second using a quadratic Gaussian classifier, features based on Line Spectral Frequencies and higher order crossings and a simple smoothing algorithm [4]. Panagiotakis et al. predicted seg- ment changes with an accuracy of 97 %, mostly within a 0.2 second interval. Classifying the resulting segments yielded a classification accuracy of 95 %. The latency caused is 3 seconds [6]. This work proposes and evaluates a segmentation algo- rithm to discriminate speech, music and noise from an au- dio stream, which can be used as a preprocessing step for an online ASR. In this context online refers to stream decod- ing with real-time requirements. Music …

Content truncated to 500 words