Low-Resource Multilingual and Zero-Shot Multispeaker TTS

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 741–751 November 20–23, 2022. ©2022 Association for Computational Linguistics 741 Low-Resource Multilingual and Zero-Shot Multispeaker TTS Florian Lux and Julia Koch and Ngoc Thang Vu University of Stuttgart florian.lux@ims.uni-stuttgart.de Abstract While neural methods for text-to-speech (TTS) have shown great advances in modeling mul- tiple speakers, even in zero-shot settings, the amount of data needed for those approaches is generally not feasible for the vast majority of the world’s over 6,000 spoken languages. In this work, we bring together the tasks of zero-shot voice cloning and multilingual low- resource TTS. Using the language agnostic meta learning (LAML) procedure and modi- fications to a TTS encoder, we show that it is possible for a system to learn speaking a new language using just 5 minutes of training data while retaining the ability to infer the voice of even unseen speakers in the newly learned lan- guage. We show the success of our proposed approach in terms of intelligibility, naturalness and similarity to target speaker using objective metrics as well as human studies and provide our code and trained models open source. 1 Introduction The applications of modern TTS systems are om- nipresent and bring major benefits in a very diverse range of tasks. For example, low-resource TTS can be used to revitalize and conserve languages with diminishing numbers of speakers (Pine et al., 2022). Other recent applications go into the di- rection of protecting the privacy of a speaker, by exchanging their voice for a different voice, while not affecting the content of what is said (Meyer et al., 2022). Even in literary studies, TTS systems can be applied to investigate perceptive aspects of poetry reading (Koch et al., 2022). However, while the first of those examples can be done with just a single speaker, the latter two require the TTS system to be able to exchange the voice of the utter- ance that is produced, which usually requires large amounts of clean multispeaker data. The same requirement exists for many other such applica- tions, which can also be seen in the rise of interest in the research community on voice-cloning tech- nologies (Wu et al., 2022; Casanova et al., 2022; Neekhara et al., 2021; Hemati and Borth, 2021; Cooper et al., 2020). The communities of speakers of low-resourced languages are thus mostly locked out of plenty of the applications that modern TTS enables. For many instances of such languages, like the Taa language, which is famous for its 83 click sounds or the Yoruba language, in which the tones bear so much meaning, that the language can be mostly whistled, it would be extremely difficult to collect the required amounts of data, and transfer learning to such unique languages is very challeng- ing. Still, we believe that a single model that speaks many languages with any voice can exhibit strong generalizing …

Content truncated to 500 words