The YiTrans Speech Translation System for IWSLT 2022 Offline Shared Task

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 158 - 168 May 26-27, 2022 c⃝2022 Association for Computational Linguistics The YiTrans End-to-End Speech Translation System for IWSLT 2022 Offline Shared Task Ziqiang Zhang1∗, Junyi Ao2,∗ 1School of Information Science and Technology, University of Science and Technology of China 2School of Data Science, The Chinese University of Hong Kong (Shenzhen) Abstract This paper describes the submission of our end-to-end YiTrans speech translation system for the IWSLT 2022 offline task, which trans- lates from English audio to German, Chinese, and Japanese. The YiTrans system is built on large-scale pre-trained encoder-decoder mod- els. More specifically, we first design a multi- stage pre-training strategy to build a multi- modality model with a large amount of labeled and unlabeled data. We then fine-tune the cor- responding components of the model for the downstream speech translation tasks. More- over, we make various efforts to improve per- formance, such as data filtering, data augmen- tation, speech segmentation, model ensemble, and so on. Experimental results show that our YiTrans system obtains a significant improve- ment than the strong baseline on three trans- lation directions, and it achieves +5.2 BLEU improvements over last year’s optimal end-to- end system on tst2021 English-German. 1 Introduction In this paper, we describe our end-to-end speech translation system YiTrans which participates in the offline tracks of the IWSLT 2022 evaluation campaign. We evaluate our systems from English to German, Chinese and Japanese. We aim at ex- ploring the pre-training methods for end-to-end systems, and bridging the quality gap with the cas- caded approaches. As self-supervised learning has been shown ef- fective in speech-to-text tasks (Baevski et al., 2020; Hsu et al., 2021; Ao et al., 2021; Bapna et al., 2021), our teams are interested in building a multi- modality pre-trained model with self-supervised approaches by leveraging large amounts of speech and text data. Inspired by SpeechT5 (Ao et al., 2021), we design a multi-stage unified-modal train- ing strategy for pre-training both the encoder and ∗Equal contribution. decoder. Our final end-to-end ST systems are built by fine-tuning the pre-trained models. This paper also tries to improve the system per- formance by exploring various techniques for the related tasks. (1) To boost the performance with ad- vanced speech segmentation (Anastasopoulos et al., 2021), we apply the pyannote toolkit (Bredin et al., 2020) and the merge algorithm from Inaguma et al. (2021) to segment the audio. Particularly, to over- come the long sentence problem in the dataset, we design a new segment algorithm. (2) Dataset is the key point for a ST system to perform well. Hence, we conduct refined data filtering and large-scale data augmentation (Jia et al., 2019). (3) We also employ progressive learning, back translation and multi-stage fine-tuning (Yang et al., 2021; Sennrich et al., 2015; Wang et al., 2020b) when fine-tuning our models. (4) Motivated by Tang et al. (2021a), we utilize joint ST and MT fine-tuning for our end- to-end ST models. (5) As comparison, we also …

Content truncated to 500 words