Weakly-supervised text-to-speech alignment confidence measure

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2042–2050, Osaka, Japan, December 11-17 2016. Weakly-supervised text-to-speech alignment confidence measure Guillaume Serri`ere, Christophe Cerisara, Dominique Fohr, Odile Mella LORIA URM 7503, 54506 Vandoeuvre-les-Nancy, France guillaume.serriere@telecomnancy.net {cerisara,fohr,mella}@loria.fr Abstract This work proposes a new confidence measure for evaluating text-to-speech alignment sys- tems outputs, which is a key component for many applications, such as semi-automatic corpus anonymization, lips syncing, film dubbing, corpus preparation for speech synthesis and speech recognition acoustic models training. This confidence measure exploits deep neural networks that are trained on large corpora without direct supervision. It is evaluated on an open-source spontaneous speech corpus and outperforms a confidence score derived from a state-of-the-art text-to-speech aligner. We further show that this confidence measure can be used to fine-tune the output of this aligner and improve the quality of the resulting alignment. 1 Introduction This work focuses on the text-to-speech alignment (T2SA) task, which consists in temporally aligning a given speech sound file with its known text transcription. The standard objective quality metric is the expected alignment error, measured in seconds and defined as L = E[|ˆt − t|], where t is the gold timestamp of a word boundary, and ˆt the corresponding timestamp estimated by the aligner (Keshet et al., 2005). Text-to-speech alignment is an important task for many applications, including: (i) Lip-syncing in cartoons production and film dubbing; (ii) Anonymization of audio corpus; (iii) Pre-processing of audio corpora for training new speech recognition systems; (iv) Indexing audio-visual corpora for browsing and querying; (v) Sampling sounds for speech synthesis; (vi) Second-language learning. We propose in this work a novel confidence measure for detecting erroneous word boundaries at the output of an existing T2SA system. Accurately detecting misplaced word boundaries is crucial to reduce post-processing costs in every previous application. For instance, only the most reliable segments may be chosen for acoustic model training and speech synthesis, and manual corrections may be limited to the less reliable boundaries for lip syncing and corpus anonymization. The proposed confidence measure is computed by a deep neural network (DNN) that is trained on a large corpus without any manually annotated word boundaries. We show on a gold corpus of French spontaneous speech that the proposed model is able to detect correct boundaries with a significantly better accuracy than the acoustic models used in the T2SA system, thanks to the acoustic features automatically captured by the deep neural model on the large unlabelled corpus. We further show that the proposed confidence measure may be used to post-process the T2SA output and improve its precision. 2 Related works Every text-to-speech aligner faces three main challenges: (i) Handling imperfect transcriptions; (ii) Sup- porting noisy acoustic conditions; (iii) Finding the globally optimal alignment on long (up to a few hours) audio files. Many solutions have been proposed to address these issues. For instance, “anchor- based” approaches (Moreno et al., 1998; de Jong et al., 2006; Hazen, 2006) automatically infer high- confidence words timestamps at …

Content truncated to 500 words