A Preliminary Study of the Application of Discrete Wavelet Transform Features in Conv-TasNet Speech Enhancement Model
1025Authors
nan
Abstract
AbstractNowadays, time-domain features have been widely used in speech enhancement (SE) networks like frequency-domain features to achieve excellent performance in eliminating noise from input utterances. This study primarily investigates how to extract information from time-domain utterances to create more effective features in speech enhancement. We present employing sub-signals dwelled in multiple acoustic frequency bands in time domain and integrating them into a unified feature set. We propose using the discrete wavelet transform (DWT) to decompose each input frame signal to obtain sub-band signals, and a projection fusion process is performed on these signals to create the ultimate features. The corresponding fusion strategy is the bi-projection fusion (BPF). In short, BPF exploits the sigmoid function to create ratio masks for two feature sources. The concatenation of fused DWT features and time features serves as the encoder output of a celebrated SE framework, fully-convolutional time-domain audio separation network (Conv-TasNet), to estimate the mask and then produce the enhanced time-domain utterances. The evaluation experiments are conducted on the VoiceBank-DEMAND and VoiceBank-QUT tasks. The experimental results reveal that the proposed method achieves higher speech quality and intelligibility than the original Conv-TasNet that uses time features only, indicating that the fusion of DWT features created from the input utterances can benefit time features to learn a superior Conv-TasNet in speech enhancement.
Keywords
Full Content
摘要 當前基於深度類神經網路架構之語音 強化模型，常使用時域特徵來加以學 習其模型參數，時域特徵如同經典的 頻域特徵一般，能夠使所得模型達到 優異的語音強化效果。基於此概念， 本研究主要是探討如何從時域的語音 中提取資訊、以在語音強化中創建更 有效的特徵。我們提出了在時域中擷 取短時間的子頻帶信號，並將它們融 合成為單一特徵。具體方法是應用離 散小波變換對每個輸入的音框信號進 行分解、以獲得子頻帶信號，並對這 些信號進行投影融合處理以創建最終 小波域特徵。對應的融合處理法稱為 雙投影融合(bi-projection fusion, BPF)法。 同時，我們將藉由離散小波轉換之融 合小波域特徵與原始時域特徵加以整 合、來學習一高效的語音強化網路： 全 卷 積 時 域 音 頻 分 離 網 路 (Conv- TasNet)，藉此來強化受雜訊干擾的語 音訊號、提升其品質與可讀性。 我 們 在 VoiceBank-DEMAND 與 VoiceBank-QUT 兩個語音強化資料集上 進行了評估實驗，初步結果表明，所 提出的方法比原始單純使用時域特徵 的 Conv-TasNet 實現了更高的客觀語音 品質和可讀性指標，表明融合小波域 特徵可以輔助原時域特徵、從輸入的 雜訊語音中學習一個更有效的 Conv- TasNet 網路、達到更佳的語音強化效 果。 Abstract Nowadays, time-domain features have been widely used in speech enhancement (SE) networks like frequency-domain features to achieve excellent performance in eliminating noise from input utterances. This study primarily investigates how to extract information from time-domain utterances to create more effective features in speech enhancement. We present employing sub-signals dwelled in multiple acoustic frequency bands in time domain and integrating them into a unified feature set. We propose using the discrete wavelet transform (DWT) to decompose each input frame signal to obtain sub-band signals, and a projection fusion process is performed on these signals to create the ultimate features. The corresponding fusion strategy is the bi-projection fusion (BPF). In short, BPF exploits the sigmoid function to create ratio masks for two feature sources. The concatenation of fused DWT features and time features serves as the encoder output of a celebrated SE framework, fully-convolutional time- domain audio separation network (Conv- TasNet), to estimate the mask and then produce the enhanced time-domain utterances. The evaluation experiments are conducted on the VoiceBank-DEMAND and VoiceBank-QUT tasks. The experimental results reveal that the proposed method achieves higher speech quality and intelligibility than the original Conv- TasNet that uses time features only, indicating that the fusion of DWT features created from the input utterances can benefit time features to learn a superior Conv-TasNet in speech enhancement. 使用離散小波轉換特徵於 Conv-TasNet 語音強化模型的初步研究 A Preliminary Study of the Application of Discrete Wavelet Transform Features in Conv-TasNet Speech Enhancement Model 陳彥同 Yan-Tong Chen 暨南大學電機系 National Chi Nan University s109323508@mail1.ncnu.edu.tw 吳宗泰 Zong-Tai Wu 暨南大學電機系 National Chi Nan University s110323503@mail1.ncnu.edu.tw 洪志偉 Jeih-Weih Hung 暨南大學電機系 National Chi Nan University jwhung@ncnu.edu.tw The 34th Conference on Computational Linguistics and Speech Processing (ROCLING 2022) Taipei, Taiwan, November 21-22, 2022. The Association for Computational Linguistics and Chinese Language Processing 92 關鍵字：語音強化、離散小波轉換、跨域、雙 投影融合、全卷積時頻分離網路 Keywords: speech enhancement, discrete wavelet transform, cross-domain, temporal speech sequence, Conv-TasNet, bi-projection fusion 1 簡介 現今的語音處理技術已成功集成到智能 3C 設 備、實現語音辨識、交互式語音聊天、智能 機器人的語音指令控制、汽車或機車的語音 控制等功能於眾多網路和多媒體視聽設備 中。然而，在拓展語音相關之應用上，仍然 存在許多關鍵性的挑戰。從信號處理的角度 來看，雜訊(noise)干擾是信號傳輸和語音處理 的 首 要 問 題 之 一 。 正 如 2021 年 暢 銷 書 (Kahneman, 2021)中所述，雜訊(noise)和偏差 (bias)是精準估測與決策上兩個主要的錯誤來 源。然而，與偏差相比，雜訊的隨機性使其 準確估測的可能性大大降低，從而加深了處 理雜訊的難度。雖然這主要是對人類判斷的 敘述，但它似乎同樣適用於基於機器之自動 語音信號處理所面臨的處境。 在對應語音中的雜訊之各種議題中，語音 強 化 (speech enhancement, SE) (Philipos C. Loizou,2013)應可謂最直接的處理機制，其針 對接收到的語音訊號加以處理、目標為提升 輸出語音訊號的訊雜比(signal-to-noise ratio, SNR)，或改善語音的品質(quality)和可讀性 (intelligibility)，使人們或機器更容易接受和理 解語音訊號。傳統的語音強化演算法主要依 賴於語音或雜訊的統計特性建構模型，然而 它們在非穩態雜訊場景中的表現通常較差， 主因之一在於非穩態的訊號特性較難以統計 形式加以精準估測。 近十年來，由於深度學習和深度神經網路 (deep neural networks, DNN)(Ian Goodfellow et al., 2016) 的理論與應用的飛快進步，學者和專 家藉由 DNN 來建構語音強化的模型，與傳統 基於統計的語音強化法相較，基於 DNN 的方 法效果通常優異許多、特別是在非穩態的雜 訊場景中。 在諸多基於 DNN 的語音強化法中，全摺積 時域音訊分離網路法 (Conv-TasNet)(Yi Luo and Nima Mesgarani, 2019)相當著名且高效，因此 廣為學者所探討並加以延伸。Conv-TasNet 採 用 編 碼 器 串 接 解 碼 器(encoder-decoder)的 架 構，並應用堆疊的一維擴張卷積塊(dilated convolutional block)來執行其中的遮罩估測網 路(mask estimation network)。 原 始 的 Conv- TasNet 在編碼器端使用可訓練之一維卷積層來 創建時域特徵，以作為編碼器輸出。而跨域 時間卷積網路法(CD-TCN)(Fu-An Chao et al., 2019)則 進一步採用頻域特徵，結合時域特徵 作為編碼器輸出。實驗結果表明，相較於使 用單一時域的 Conv-TasNet，使用跨域特徵 （時域與頻域）的CD-TCN達到更佳的語音強 化效能。 在本研究中，我們參考了 CD-TCN 的方法 脈絡，嘗試通過添加另一個特徵源來改進 Conv-TasNet，而這個特徵源來自輸入之音框 訊 號 的 離 散 小 波 變 換 (discrete wavelet transform, DWT)(Stephane Mallat,2019)。我們 使用 DWT 創建子頻帶特徵，然後將這些子頻 …
Content truncated to 500 words
Labeled Text
AbstractNowadays, time-domain features have been widely used in speech enhancement (SE) networks like frequency-domain features to achieve excellent performance in eliminating noise from input utterances. This study primarily investigates how to extract information from time-domain utterances to create more effective features in speech enhancement. We present employing sub-signals dwelled in multiple acoustic frequency bands in time domain and integrating them into a unified feature set. We propose using the discrete wavelet transform (DWT) to decompose each input frame signal to obtain sub-band signals, and a projection fusion process is performed on these signals to create the ultimate features. The corresponding fusion strategy is the bi-projection fusion (BPF). In short, BPF exploits the sigmoid function to create ratio masks for two feature sources. The concatenation of fused DWT features and time features serves as the encoder output of a celebrated SE framework, fully-convolutional time-domain audio separation network (Conv-TasNet), to estimate the mask and then produce the enhanced time-domain utterances. The evaluation experiments are conducted on the VoiceBank-DEMAND and VoiceBank-QUT tasks. The experimental results reveal that the proposed method achieves higher speech quality and intelligibility than the original Conv-TasNet that uses time features only, indicating that the fusion of DWT features created from the input utterances can benefit time features to learn a superior Conv-TasNet in speech enhancement.
Publication Details
- Paper ID:
- 1025
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan