Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Features

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6858 - 6868 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Features Florian Lux and Ngoc Thang Vu Institute for Natural Language Processing University of Stuttgart florian.lux@ims.uni-stuttgart.de Abstract While neural text-to-speech systems perform remarkably well in high-resource scenarios, they cannot be applied to the majority of the over 6,000 spoken languages in the world due to a lack of appropriate training data. In this work, we use embeddings derived from articu- latory vectors rather than embeddings derived from phoneme identities to learn phoneme rep- resentations that hold across languages. In con- junction with language agnostic meta learning, this enables us to fine-tune a high-quality text- to-speech model on just 30 minutes of data in a previously unseen language spoken by a previ- ously unseen speaker. 1 Introduction The advance of deep learning (Vaswani et al., 2017; Goodfellow et al., 2014) has enabled great im- provements in the field of Text-to-Speech (TTS). (Towards-)end-to-end models, such as Tacotron 2 (Wang et al., 2017; Shen et al., 2018), Trans- formerTTS (Li et al., 2019b), FastSpeech 2 (Ren et al., 2019, 2020), FastPitch (Ła´ncucki, 2021) and many more famous instances (e.g. Arık et al. (2017) and Prenger et al. (2019)) allow for speech synthesis with unprecedented quality and controlla- bility. The models mentioned here rely on vocoders, such as WaveNet (van den Oord et al., 2016), MelGAN (Kumar et al., 2019), Parallel Wave- GAN (Yamamoto et al., 2020) or HiFi-GAN (Kong et al., 2020) to turn the parametric representations that they produce into waveforms. Recently pro- posed models even include some with the ability to go directly to the waveform from a grapheme or phoneme input sequence, such as EATS (Donahue et al., 2020) or VITS (Kim et al., 2021). While these methods all perform remarkably well if given enough data, cross-lingual use of data remains a key challenge in TTS. Most mod- ern methods are limited to languages and domains that are rich in resources, which over 6,000 lan- guages are not. Attempts at reducing the required resources in a target language by making use of transfer learning from multilingual data have been made by Azizah et al. (2020); Xu et al. (2020); Chen et al. (2019). The mismatch of input spaces however requires complex architectural changes, which limits their ability to be used in conjunction with other modern TTS architectures. Attempts at fixing the issue of having to transfer knowledge from a source to a target by just jointly training on a mixed set of more and less resource rich languages have been made by He et al. (2021); de Korte et al. (2020); Yang and He (2020), which requires com- plex training procedures. In this work, we will also attempt to transfer knowledge from a set of high resource languages to a low resource language. We fix previous shortcomings by 1) using a linguisti- cally motivated representation …

Content truncated to 500 words