Speech enhancement based on the integration of fully convolutional network, temporal lowpass filtering and spectrogram masking

Speech enhancement based on the integration of fully convolutional network, temporal lowpass filtering and spectrogram masking Kuan-Yi Liu1, Syu-Siang Wang2, Yu Tsao2, Jeih-weih Hung1 1National Chi Nan University, Taiwan 2Academia Sinica, Taiwan s106323508@mail1.ncnu.edu.tw, sypdbhee@gmail.com, yu.tsao@citi.sinica.edu.tw, jwhung@ncnu.edu.tw Abstract In this study, we focus on the issue of noise distortion in speech signals, and develop two novel unsupervised speech enhancement algorithms including temporal lowpass filtering (TLP) and relative-to-maximum masking (RMM). Both of these two algorithms are conducted on the magnitude spectrogram of speech signals. TLP uses a simple moving-average filter to emphasize the low modulation frequencies of speech signals, which are believed to contain richer linguistic information and exhibit higher signal-to-noise ratios (SNR). Comparatively, in RMM we apply a mask that is directly multiplied with the speech spectrogram in a point- wise manner, and the used masking value is directly proportional to the magnitude of each temporal-frequency (T-F) point in the spectrogram. The preliminary experiments conducted on a subset of TIMIT database show that the two novel methods can promote the quality of noise- corrupted speech signals significantly, and both of them can be integrated with a well-known supervised speech enhancement scenario, namely fully convolutional network, to achieve even better perceptual speech quality values. Keywords: temporal lowpass filtering, relative-to-maximum masking, moving-average filter, speech enhancement 1. Introduction Nowadays the technologies of communication have been developed quite quickly and they have changed and influenced our life a lot. In particular, speech communication such as speech signal transmission and reception through a wired or a wireless network, has been a widespread use in our daily life [1-2]. Therefore, high speech quality and intelligibility during communication gradually becomes a prerequisite. However, in the transmission environment of speech signals, there exist lots of distortions, such as additive noise, channel mismatch and reverberation, which inevitably decrease the speech signal quality/intelligibility seriously. To overcome these distortions in speech communication, a lot of researchers in recent decades have been devoted to developing speech enhancement (SE) techniques. These SE algorithms can be classified based on whether a learning/training process is involved. For example, if the noise statistics in spectral-subtractive The 2019 Conference on Computational Linguistics and Speech Processing ROCLING 2019, pp. 226-240 ©The Association for Computational Linguistics and Chinese Language Processing 226 SE algorithms [3-6] and the basis of the clean signal subspace in subspace SE algorithms [7-9] are learned via a training set with explicit labels, then the corresponding SE methods are supervised. Comparatively, in the unsupervised methods, such as spectral subtraction (SS) [10], Wiener filtering [11], short-time spectral amplitude (STSA) estimation [12] and short-time log- spectral amplitude estimation (logSTSA) [13], does not employ prior information about speech and/or noise. In this study, we develop two novel learning-free SE methods. One is called temporal low- pass filtering (TLF) and the other is relative-to-maximum masking (RMM). Briefly speaking, TLF borrows the idea of Mod-WD [14], a learning-free SE method, while it can be implemented significantly more simply than Mod-WD, and the mask used in RMM is totally data-driven, viz. it is …

Content truncated to 500 words