KIT’s Multilingual Speech Translation System for IWSLT 2023

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 113–122 July 13-14, 2023 c⃝2023 Association for Computational Linguistics KIT’s Multilingual Speech Translation System for IWSLT 2023 Danni Liu, Thai Binh Nguyen, Sai Koneru, Enes Yavuz Ugan, Ngoc-Quan Pham, Tuan-Nam Nguyen, Tu Anh Dinh, Carlos Mullov, Alexander Waibel, Jan Niehues Karlsruhe Institute of Technology firstname.lastname@kit.edu Abstract Many existing speech translation benchmarks focus on native-English speech in high-quality recording conditions, which often do not match the conditions in real-life use-cases. In this paper, we describe our speech translation sys- tem for the multilingual track of IWSLT 2023, which focuses on the translation of scientific conference talks. The test condition features accented input speech and terminology-dense contents. The tasks requires translation into 10 languages of varying amounts of resources. In absence of training data from the target do- main, we use a retrieval-based approach (kNN- MT) for effective adaptation (+0.8 BLEU for speech translation). We also use adapters to easily integrate incremental training data from data augmentation, and show that it matches the performance of re-training. We observe that cascaded systems are more easily adapt- able towards specific target domains, due to their separate modules. Our cascaded speech system outperforms its end-to-end counterpart on scientific talk translation, although their per- formance remains similar on TED talks. 1 Introduction This paper summarizes Karlsruhe Institute of Tech- nology’s speech translation system for the multilin- gual track of IWSLT 2023 (Agarwal et al., 2023). In this track, the task is to translate scientific talks in English into 10 languages: Arabic (ar), Chinese (zh), Dutch (nl), French (fr), German (de), Japanese (ja), Persian/Farsi (fa), Portuguese (pt), Russian (ru), Turkish (tr). The talks are from presentations in the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022). Translating scientific talks presents several chal- lenges. On the source side, most speakers are non-native, and the recording conditions often vary. This requires acoustic robustness to accents and noise. On the target side, domain-specific termi- nologies are frequently used, calling for accurate translation of these words that rarely occur in the training data. The styles of the talks, e.g. formal- ity, also differ from other domains. As no training data from the same domain is provided, effective few-shot or zero-shot adaptation is crucial. As the task focuses on one-to-many translation, it is also an interesting testbed for whether mul- tilinguality improves speech translation quality. For text-to-text translation, the gain from multi- linguality is mostly concentrated in many-to-one translation (Aharoni et al., 2019; Fan et al., 2021), i.e., multilinguality on the source side. In con- trast, for X-to-many translation, it remains unclear whether incorporating more target languages im- proves translation quality. In this system description paper, we present cascaded and end-to-end systems for the English- to-many speech translation task. We lever- age pretrained models, including WavLM (Chen et al., 2022), mBART50 (Tang et al., 2020), and DeltaLM (Ma et al., 2021). The systems do not use additional data beyond the allowed corpora, and therefore fall under …

Content truncated to 500 words