NVIDIA NeMo Offline Speech Translation Systems for IWSLT 2022

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 225 - 231 May 26-27, 2022 c⃝2022 Association for Computational Linguistics NVIDIA NeMo Offline Speech Translation Systems for IWSLT 2022 Oleksii Hrinchuk*, Vahid Noroozi, Abhinav Khattar, Anton Peganov, Sandeep Subramanian, Somshubra Majumdar, Oleksii Kuchaiev NVIDIA, Santa Clara, CA Abstract This paper provides an overview of NVIDIA NeMo’s speech translation systems for the IWSLT 2022 Offline Speech Translation Task. Our cascade system consists of 1) Conformer RNN-T automatic speech recognition model, 2) punctuation-capitalization model based on pre- trained T5 encoder, 3) ensemble of Transformer neural machine translation models fine-tuned on TED talks. Our end-to-end model has less parameters and consists of Conformer encoder and Transformer decoder. It relies on the cas- cade system by re-using its pre-trained ASR encoder and training on synthetic translations generated with the ensemble of NMT models. Our En→De cascade and end-to-end systems achieve 29.7 and 26.2 BLEU on the 2020 test set correspondingly, both outperforming the previous year’s best of 26 BLEU. 1 Introduction We participate in the IWSLT 2022 Offline Speech Translation Task (Anastasopoulos et al., 2022) for English→German and English→Chinese. Due to the limited amount of direct speech translation (ST) data, we mostly focused on building a strong cas- cade pipeline structured as follows: • ASR model with Conformer (Gulati et al., 2020b) encoder and RNN-T (Graves, 2012) decoder trained with SpecAugment (Park et al., 2019) which transforms input audio into lower-cased text without punctuation. • Punctuation-capitalization (PC) model with T5 (Raffel et al., 2019) encoder and classifica- tion head which transforms normalized ASR output into standard English text, more suit- able for NMT model. • Ensemble of 4 NMT Transformers (Vaswani et al., 2017) trained with back-translation and *Correspondence to: ohrinchuk@nvidia.com right-to-left distillation and fine-tuned on TED talks which translates English text into target language. We also trained end-to-end models capitalizing on the pre-trained ASR encoder and synthetic trans- lations obtained with the ensemble of NMT mod- els. Our best end-to-end model consisting of Con- former encoder and Transformer decoder lags be- hind the best cascade by 2.7 BLEU on average, however, it might be preferred for some scenarios of limited resources or latency requirements. Our systems are open-sourced as part of NVIDIA NeMo1 framework (Kuchaiev et al., 2019). 2 Data In this section, we describe the datasets used for training (Table 1). For evaluation, we used the development sets of Must-C v2, as well as the test sets from past IWSLT competitions. ASR For training our ASR model, we used Lib- riSpeech (Panayotov et al., 2015), Mozilla Com- mon Voice v6.1 (Ardila et al., 2019), TED-LIUM v3 (Hernandez et al., 2018), VoxPopuli v2 (Wang et al., 2021a), all available speech-to-English data from Must-C v2 (Cattoni et al., 2021) En-De/Zh/Ja datasets, ST-TED (Jan et al., 2018), and clean por- tion of Europarl-ST (Iranzo-Sánchez et al., 2020). PC For training our punctuation-capitalization (PC) model, we combined 268M sentences from Europarl (Koehn, 2005), RAPID (Rozis and Skadin, š, 2017), TED (Cettolo et …

Content truncated to 500 words