I2R’s End-to-End Speech Translation System for IWSLT 2023 Offline Shared Task

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 202–210 July 13-14, 2023 c⃝2023 Association for Computational Linguistics I2R’s End-to-End Speech Translation System for IWSLT 2023 Offline Shared Task Muhammad Huzaifah, Kye Min Tan, Richeng Duan Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore Abstract This paper describes I2R’s submission to the of- fline speech translation track for IWSLT 2023. We focus on an end-to-end approach for trans- lation from English audio to German text, one of the three available language directions in this year’s edition. The I2R system leverages on pretrained models that have been exposed to large-scale audio and text data for our base model. We introduce several stages of addi- tional pretraining followed by fine-tuning to adapt the system for the downstream speech translation task. The strategy is supplemented by other techniques such as data augmentation, domain tagging, knowledge distillation, and model ensemble, among others. We evaluate the system on several publicly available test sets for comparison. 1 Introduction Historically, speech translation (ST) has involved combining automatic speech recognition (ASR) and machine translation (MT) systems in a cas- cade. The ASR system would transcribe speech signals into text in the source language, and the MT system would then translate this text into the target language. However, recent developments in deep learning have made it possible to use an end-to-end speech translation model (Bérard et al., 2016; Weiss et al., 2017), which directly trans- lates speech in the source language into text in the target language, without relying on intermedi- ate symbolic representations. This approach offers the advantages of lower latency and avoids error propagation. While cascaded models initially out- performed end-to-end models, recent results from IWSLT campaigns (Le et al., 2020; Bentivogli et al., 2021; Anastasopoulos et al., 2022) have shown that the performance of end-to-end models is now ap- proaching that of cascaded solutions. Large pretrained models (Lewis et al., 2020; Conneau et al., 2021; Raffel et al., 2020) have be- come a prevalent basis for speech and language pro- cessing work (Ma et al., 2021; Chen et al., 2022a). Through the utilization of pretrained models and subsequent finetuning using a small amount of la- beled data, many tasks have exhibited significant improvements in performance (Baevski et al., 2020; Hsu et al., 2021; Guillaume et al., 2022; Navarro et al., 2022), some even reaching state-of-the-art results. In this work, we describe our end-to-end system for the Offline Speech Translation Task at IWSLT 2023 (Agarwal et al., 2023) in the English-German (En-De) language direction. The current year’s task not only includes the traditional TED talk evalu- ation set translated from English to German, but also introduces two additional test sets consisting of ACL presentations, press conferences and in- terviews (EMPAC), which are more complex and challenging. Furthermore, this year’s constrained data track allows less data than previous years. Our team enhances the end-to-end ST system within the context of the pretrain-finetune paradigm. We in- troduce several pretraining stages before finetuning …

Content truncated to 500 words