Revisiting Over-Smoothness in Text to Speech

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8197 - 8213 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Revisiting Over-Smoothness in Text to Speech Yi Ren Zhejiang University rayeren@zju.edu.cn Xu Tan Microsoft Research xuta@microsoft.com Tao Qin Microsoft Research taoqin@microsoft.com Zhou Zhao∗ Zhejiang University zhaozhou@zju.edu.cn Tie-Yan Liu Microsoft Research tyliu@microsoft.com Abstract Non-autoregressive text to speech (NAR-TTS) models have attracted much attention from both academia and industry due to their fast gen- eration speed. One limitation of NAR-TTS models is that they ignore the correlation in time and frequency domains while generating speech mel-spectrograms, and thus cause blurry and over-smoothed results. In this work, we re- visit this over-smoothing problem from a novel perspective: the degree of over-smoothness is determined by the gap between the complexity of data distributions and the capability of mod- eling methods. Both simplifying data distri- butions and improving modeling methods can alleviate the problem. Accordingly, we first study methods reducing the complexity of data distributions. Then we conduct a comprehen- sive study on NAR-TTS models that use some advanced modeling methods. Based on these studies, we find that 1) methods that provide ad- ditional condition inputs reduce the complexity of data distributions to model, thus alleviating the over-smoothing problem and achieving bet- ter voice quality. 2) Among advanced model- ing methods, Laplacian mixture loss performs well at modeling multimodal distributions and enjoys its simplicity, while GAN and Glow achieve the best voice quality while suffering from increased training or model complexity. 3) The two categories of methods can be com- bined to further alleviate the over-smoothness and improve the voice quality. 4) Our experi- ments on the multi-speaker dataset lead to sim- ilar conclusions as above and providing more variance information can reduce the difficulty of modeling the target data distribution and al- leviate the requirements for model capacity. 1 Introduction Non-autoregressive text to speech (NAR-TTS) models (Ren et al., 2019, 2020; Peng et al., 2020; Vainer and Dušek, 2020; Ła´ncucki, 2020; Kim ∗ Corresponding author et al., 2020; Miao et al., 2020) have shown much faster inference speed than their autoregressive counterparts (Wang et al., 2017; Shen et al., 2018; Ping et al., 2018), while achieving comparable or even better voice quality (Ren et al., 2019, 2020). The text-to-speech mapping can be formulated as a conditional distribution P(y|x) where x and y are the text and speech sequences, respectively. Text- to-speech mapping is a one-to-many mapping prob- lem (Wang et al., 2017), since multiple possible speech sequences correspond to a text sequence due to speech variations such as pitch, duration and prosody. Furthermore, speech mel-spectrograms are strongly correlated in time and frequency di- mensions (see Section 2.1 for detailed analyses). Therefore, P(y|x) is actually a dependent and mul- timodal distribution (Ling et al., 2013; Zen and Senior, 2014)1. Early non-autoregressive TTS models (Ren et al., 2019; Peng et al., 2020) use mean absolute error (MAE) or mean square error (MSE) as loss func- tion to model …

Content truncated to 500 words