Sound Signal Processing with Seq2Tree Network

Sound Signal Processing with Seq2Tree Network Weicheng Ma†, Kai Cao‡, Zhaoheng Ni♦, Peter Chin†, Xiang Li‡ †Boston University ‡Cambia Health Solutions ♦City University of New York †{wcma, spchin}@bu.edu, ♦zni@gradcenter.cuny.edu ‡{kai.cao, xiang.li}@cambiahealth.com Abstract Long Short-Term Memory (LSTM) and its variants have been the standard solution to sequential data processing tasks because of their ability to preserve previous information weighted on distance. This feature provides the LSTM family with additional information in predictions, compared to regular Recurrent Neural Networks (RNNs) and Bag-of-Words (BOW) models. In other words, LSTM networks assume the data to be chain-structured. The longer the distance between two data points, the less related the data points are. However, this is usually not the case for real multimedia signals including text, sound and music. In real data, this chain-structured dependency exists only across meaningful groups of data units but not over single units directly. For example, in a prediction task over sound signals, a meaningful word could give a strong hint to its following word as a whole but not the first phoneme of that word. This undermines the ability of LSTM networks in modeling multimedia data, which is pattern-rich. In this paper we take advantage of Seq2Tree network, a dynamically extensible tree-structured neural network architecture which helps solve the problem LSTM networks face in sound signal processing tasks—the unbalanced connections among data units inside and outside semantic groups. Experiments show that Seq2Tree network outperforms the state-of-the-art Bidirectional LSTM (BLSTM) model on a signal and noise separation task (CHiME Speech Separation and Recognition Challenge). Keywords: Deep Learning, Tree Model, Dynamic Neural Network 1. Introduction Recent RNN-based approaches are achieving high perfor- mance in speech processing tasks, including but not limited to the signal and noise separation task (Erdogan et al., 2015; Zhu and Vogel-Heuser, 2014; Wu et al., 2015; Barker et al., 2015). The underlying hypothesis is that the energy in each frequency bin over a period of time is continuous and pre- dictable. However, in real life scenes noises can break in at any time and intertwine with the sound signal with no pre- dictable pattern, which undermines these models’ ability to predict the distribution of noise over frequency bins. To address the problem of finding correct boundaries of noises, some variants of the original LSTM network are used. The current state-of-the-art system on this task ap- plies BLSTM, which tries to bound noises by foreseeing future information (Erdogan et al., 2015; Weninger et al., 2015; Grais et al., 2014). Nevertheless, information from the future also contains more distant sound signals, which does not solve the signal superposition problem. Further- more, we believe the future for speech processing should be dominated by real-time speech processing techniques, which BLSTM models are not able to handle. What’s more important, phonemes in sound signals make no sense if not combined into “words”, which are not found in noise signals. So, the sound waveforms should not be understood as a chain of phonemes, but rather on a “word” level. This leads to a …

Content truncated to 500 words