The Xiaomi AI Lab’s Speech Translation Systems for IWSLT 2023 Offline Task, Simultaneous Task and Speech-to-Speech Task

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 411–419 July 13-14, 2023 c⃝2023 Association for Computational Linguistics The Xiaomi AI Lab’s Speech Translation Systems for IWSLT 2023 Offline Task, Simultaneous Task and Speech-to-Speech Task Wuwei Huang1∗† Mengge Liu2∗‡ Xiang Li1 Yanzhi Tian2‡ Fengyu Yang1 Wen Zhang1 Yuhang Guo2 Jinsong Su3 Jian Luan1 Bin Wang1 1Xiaomi AI Lab, Beijing, China 2Beijing Institute of Technology, Beijing, China 3Xiamen University, Xiamen, Fujian, China. {huangwuwei,lixiang21,yangfengyu1,zhangwen17,luanjian,wangbin11}@xiaomi.com {liumengge,tianyanzhi,guoyuhang}@bit.edu.cn jssu@xmu.edu.cn Abstract This system description paper introduces the systems submitted by Xiaomi AI Lab to the three tracks of the IWSLT 2023 Evaluation Campaign, namely the offline speech transla- tion (Offline-ST) track, the offline speech-to- speech translation (Offline-S2ST) track, and the simultaneous speech translation (Simul-ST) track. All our submissions for these three tracks only involve the English-Chinese language di- rection. Our English-Chinese speech transla- tion systems are constructed using large-scale pre-trained models as the foundation. Specifi- cally, we fine-tune these models’ correspond- ing components for various downstream speech translation tasks. Moreover, we implement sev- eral popular techniques, such as data filtering, data augmentation, speech segmentation, and model ensemble, to improve the system’s over- all performance. Extensive experiments show that our systems achieve a significant improve- ment over the strong baseline systems in terms of the automatic evaluation metric. 1 Introduction We submit an end-to-end offline speech transla- tion system, a cascaded offline speech-to-speech translation system, and an end-to-end simultane- ous interpretation system to the Offline-ST track, Offline-S2ST track, and Simul-ST track, respec- tively. This paper provides a detailed description of the three systems we submit. There are two commonly used solutions for speech translation models: the end-to-end approach and the cascaded approach. The cascaded system uses a pipeline where an automatic speech recogni- tion (ASR) system is followed by a machine transla- tion (MT) system. The ASR system first transcribes the speech utterances in the source language into ∗Equal contribution. †Crossponding Author. ‡ The work was done during the author’s internship at Xiaomi. text in the same language, and then the MT model translates the ASR output into text in the target language. In contrast, the end-to-end ST system directly translates speech utterances in the source language into text in the target language. The scarcity of training data makes end-to-end systems still slightly inferior in translation qual- ity to cascaded systems, which suffer from er- ror propagation and information loss (Sperber and Paulik, 2020). Cascaded systems continue to domi- nate the systems submitted at IWSLT in previous years (Anastasopoulos et al., 2022, 2021; Ansari et al., 2020). However, with the rapid develop- ment of pre-training technology, a large number of large-scale pre-training models suitable for various modalities, such as speech (Baevski et al., 2020; Hsu et al., 2021; Tang et al., 2022) and text (Liu et al., 2020), have emerged. Therefore, end-to-end ST systems have gradually attracted attention from both the academic and industrial communities in recent years. In our submission, we have opted for an end-to-end approach to establish the ST system. …

Content truncated to 500 words