Investigating Inter- and Intra-speaker Voice Conversion using Audiobooks

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7305–7313 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7305 Investigating Inter- and Intra-speaker Voice Conversion using Audiobooks Aghilas Sini, Damien Lolive, Nelly Barbot, Pierre Alain Univ Rennes, CNRS, IRISA 6 Rue de Kerampont CS 80518, F-22305 Lannion, France Abstract Audiobook readers play with their voices to emphasize some text passages, highlight discourse changes or significant events, or in order to make listening easier and entertaining. A dialog is a central passage in audiobooks where the reader applies significant voice transformation, mainly prosodic modifications, to realize character properties and changes. However, these intra-speaker modifications are hard to reproduce with simple text-to-speech synthesis. The manner of vocalizing characters involved in a given story depends on the text style and differs from one speaker to another. In this work, this problem is investigated through the prism of voice conversion. We propose to explore modifying the narrator’s voice to fit the context of the story, such as the character who is speaking, using voice conversion. To this end, two complementary experiments are designed: the first one aims to assess the quality of our Phonetic PosteriorGrams (PPG)-based voice conversion system using parallel data. Subjective evaluations with naive raters are conducted to estimate the quality of the signal generated and the speaker similarity. The second experiment applies an intra-speaker voice conversion, considering narration passages and direct speech passages as two distinct speakers. Data are then non parallel and the dissimilarity between character and narrator is subjectively measured. Keywords: Voice Conversion, Expressive Speech, Audiobook, PPG 1. Introduction Audiobook generation is a hard task for which speech synthesis systems need to be able to mimic both the narrator and also different characters, with sometimes very different characteristics. Usually, audiobook read- ers play with their voices to emphasize some text passages, highlight discourse changes or significant events, or in order to make listening easier and enter- taining. For instance, dialogs are central passages in audiobooks where the reader applies significant voice transformation, mainly prosodic modifications, to real- ize character properties and changes. However, these intra-speaker modifications are hard to reproduce with simple text-to-speech synthesis. The manner of vocal- izing characters involved in a given story depends on the text style and differs from one speaker to another. In this work, this problem is investigated through the prism of voice conversion. Voice Conversion (VC) aims to transform the speech of a source speaker, without changing the linguistic in- formation, such that it seems to be uttered by a target speaker. During the last decade, VC techniques have known a large development due to the flexibility of the recent synthesis technologies based on advanced deep learning methods. They allow the design of new appli- cations requiring voice modification or a large variety of voices. In 2016, a bi-annual Voice Conversion Challenge (VCC) has been launched to provide a common frame- work to compare state-of-the-art VC techniques on a same task and a …

Content truncated to 500 words