ZJU’s IWSLT 2021 Speech Translation System

Proceedings of the 18th International Conference on Spoken Language Translation, pages 144–148 Bangkok, Thailand (Online), August 5–6, 2021. ©2021 Association for Computational Linguistics 144 ZJU’s IWSLT 2021 Speech Translation System Linlin Zhang Zhejiang University 11921133@zju.edu.cn Abstract In this paper, we describe Zhejiang Univer- sity’s submission to the IWSLT2021 Multi- lingual Speech Translation Task. This task focuses on speech translation (ST) research across many non-English source languages. Participants can decide whether to work on constrained systems or unconstrained systems which can use external data. We create both cascaded and end-to-end speech translation constrained systems, using the provided data only. In the cascaded approach, we combine Conformer-based automatic speech recogni- tion (ASR) with the Transformer-based neural machine translation (NMT). Our end-to-end direct speech translation systems use ASR pre- trained encoder and multi-task decoders. The submitted systems are ensembled by different cascaded models. 1 Introduction In this paper, we introduce our submission to the IWSLT2021 Multilingual Speech Translation Task. This task focuses on speech translation (ST) re- search across many non-English source languages. Multilingual models enable transfer from related tasks, which is particularly important for low- resource languages; however, parallel data between two otherwise high-resource languages can often be rare, making multilingual translation and zero- shot translation important for many resource set- tings. The task provides data for two conditions (Salesky et al., 2021): supervised, and zero-shot, including speech and transcripts for four languages (Spanish, French, Portuguese, Italian) and transla- tions in a subset of five languages (English, Span- ish, French, Portuguese, Italian). At evaluation time, using the provided speech in the four source languages, participants submit the generated trans- lations in both English and Spanish. In the cascaded approach, we use a Conformer (Gulati et al., 2020) model for ASR for every lan- guage. For the MT component, we use a unified Transformer model for all language pairs. As previ- ous works (Gangi et al., 2019; Bahar et al., 2020), we use both the clean and noisy speech transcripts, back translation data, and the mask noisy trick. For the end-to-end direct speech translation, we also created a Transformer-based model. To obtain the best possible translation quality, we apply data augmentation on audio files, make a multitask de- coding for incorporating the ASR task (Weiss et al., 2017). We tried various experimental parameter settings and different architectures, and finally submitted an ensembled cascaded system. 2 Cascaded Speech Translation As the task provides speech and transcripts for four languages (Spanish, French, Portuguese, Italian) and translations in a subset of five languages (En- glish, Spanish, French, Portuguese, Italian). Zero- shot language pairs have ASR data released for training but not translations. Cascades of separately trained automatic speech recognition and machine translation (MT) models can leverage all of these data sources. 2.1 Automatic Speech Recognition We only focus on sequence-to-sequence ASR mod- els. We firstly used a Transformer-based (Vaswani et al., 2017) model on FAIRSEQ1. Our transformer- based models presented as Synnaeve et al. (2019) consist of 2 1-D convolutional subsampler lay- ers and 12 transformer …

Content truncated to 500 words