Audiobook Dialogues as Training Data for Conversational Style Synthetic Voices

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1047–1053 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 1047 Audiobook Dialogues as Training Data for Conversational Style Synthetic Voices Liisi Piits, Hille Pajupuu, Heete Sahkai, Rene Altrov, Liis Ermus, Kairi Tamuri, Indrek Hein, Meelis Mihkla, Indrek Kiissel, Egert Männisalu, Kristjan Suluste, Jaan Pajupuu Department of Speech Research and Technology, Institute of the Estonian Language, Tallinn, Estonia {liisi.piits}@eki.ee Abstract Synthetic voices are increasingly used in applications that require a conversational speaking style, raising the question as to which type of training data yields the most suitable speaking style for such applications. This study compares voices trained on three corpora of equal size recorded by the same speaker: an audiobook character speech (dialogue) corpus, an audiobook narrator speech corpus, and a neutral-style sentence-based corpus. The voices were trained with three text-to-speech synthesisers: two hidden Markov model-based synthesisers and a neural synthesiser. An evaluation study tested the suitability of their speaking style for use in customer service voice chatbots. Independently of the synthesiser used, the voices trained on the character speech corpus received the lowest, and those trained on the neutral-style corpus the highest scores. However, the evaluation results may have been confounded by the greater acoustic variability, less balanced sentence length distribution, and poorer phonemic coverage of the character speech corpus, especially compared to the neutral-style corpus. Therefore, the next step will be the creation of a more uniform, balanced, and representative audiobook dialogue corpus, and the evaluation of its suitability for further conversational-style applications besides customer service chatbots. Keywords: speech corpus, text-to-speech synthesis, speaking style, synthetic speech evaluation, chatbots, Estonian, GeMAPS 1. Introduction Synthetic voices are increasingly used in applications that require a conversational speaking style, such as voice- enabled chatbots and automatic dubbing. This raises the question as to which type of training data yields the most suitable synthetic voices for such purposes. As a general principle, a certain amount of data that represents the desired speaking style is required in order to achieve a corresponding style in synthesis. This means that a sufficient amount of conversational data should be used to achieve a conversational style synthetic voice. So far, relatively few attempts have been made to use genuinely spontaneous conversational training data (see e.g. Andersson et al., 2012; Székely et al., 2019; Yan et al., 2021). The acquisition of high-quality conversational data is a challenge, especially for low resource languages, and its processing is time-consuming, and requires novel technical solutions. Moreover, genuine spontaneous data may not be suitable for certain conversational style applications. For instance, Andersson et al. (2012) found that conversational utterances synthesised with voices trained on spontaneous data (using techniques based on hidden Markov models (HMM)) were perceived as more natural and suitable only when they contained discourse markers and filled pauses. In cases of more fluent conversational style utterances, listeners preferred voices that were based on standard read-aloud training data. A further question is how human the …

Content truncated to 500 words