VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 993–1003 August 1–6, 2021. ©2021 Association for Computational Linguistics 993 VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation Changhan Wang⋆, Morgane Rivi`ere⋆, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, Emmanuel Dupoux Facebook AI {changhan,mriviere,annl,annewu,talnikar,dhaziza, marywilliamson,juancarabina,dpx}@fb.com Abstract We introduce VoxPopuli, a large-scale multi- lingual corpus providing 400K hours of un- labeled speech data in 23 languages. It is the largest open data to date for unsuper- vised representation learning as well as semi- supervised learning. VoxPopuli also con- tains 1.8K hours of transcribed speeches in 15 languages and their aligned oral inter- pretations into 15 target languages totaling 17.3K hours. We provide speech recogni- tion (ASR) baselines and validate the versa- tility of VoxPopuli unlabeled data in semi- supervised ASR and speech-to-text transla- tion under challenging out-of-domain settings. The corpus is available at https://github. com/facebookresearch/voxpopuli. 1 Introduction Recent progress in speech-to-text tasks such as automatic speech recognition (ASR) and speech translation (ST) has been achieved by the devel- opment and application of unsupervised speech pre-training methods (Oord et al., 2018; Schnei- der et al., 2019; Baevski et al., 2020; Conneau et al., 2020; Wu et al., 2020; Nguyen et al., 2020), with semi-supervised learning (self-training) (Kahn et al., 2020a; Pino et al., 2020; Zhang et al., 2020b; Xu et al., 2020) or a combination of both meth- ods (Xu et al., 2020). This line of research lever- ages large amounts of unlabeled English speech data (Kahn et al., 2020b) that enable improve- ments in English ASR or out-of-English ST. Large amounts of multilingual audio data are needed in order to achieve similar progress for multilingual ASR and ST. Similarly, most ASR and ST research is currently conducted on the LibriSpeech (Panay- otov et al., 2015) and MuST-C benchmarks (Cattoni et al., 2020; Di Gangi et al., 2019). As a result, the ⋆Equal contribution. research community has been mostly focused on speech-to-text tasks with English as input. While multilingual ASR (Pratap et al., 2020; Ardila et al., 2020) and ST datasets (Wang et al., 2020b; Iranzo- S´anchez et al., 2020) have recently been made avail- able, the amount of data available quickly drops beyond the top few high-resource languages. Simultaneous speech translation (interpretation) has witnessed a resurgence with the applications of end-to-end encoder-decoder models. Most of the recent studies focus on text output and leverage ST corpora that are translated offline in the writ- ten form. There are differences, however, between translationese and interpretese (Sridhar et al., 2013; He et al., 2016), where interpreters develop a vari- ety of strategies to improve simultaneity. Models trained on translation corpora are unlikely to learn from these interpretation skills to achieve better quality-latency trade-offs. Finally, there has been little research (Jia et al., 2019; Tjandra et al., 2019; Zhang et al., 2020a) into speech output due to the lack of open data. …

Content truncated to 500 words