JHU IWSLT 2023 Dialect Speech Translation System Description

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 283–290 July 13-14, 2023 c⃝2023 Association for Computational Linguistics JHU IWSLT 2023 Dialect Speech Translation System Description Amir Hussein† Cihan Xiao† Neha Verma† Thomas Thebaud† Matthew Wiesner‡ Sanjeev Khudanpur†‡ †Center for Language and Speech Processing, and ‡ Human Language Technology Center of Excellence, Johns Hopkins University {ahussei6, cxiao7, nverma7, tthebau1, wiesner, khudanpur}@jhu.edu Abstract This paper presents JHU’s submissions to the IWSLT 2023 dialectal and low-resource track of Tunisian Arabic to English speech transla- tion. The Tunisian dialect lacks formal orthog- raphy and abundant training data, making it challenging to develop effective speech trans- lation (ST) systems. To address these chal- lenges, we explore the integration of large pre- trained machine translation (MT) models, such as mBART and NLLB-200 in both end-to-end (E2E) and cascaded speech translation (ST) sys- tems. We also improve the performance of au- tomatic speech recognition (ASR) through the use of pseudo-labeling data augmentation and channel matching on telephone data. Finally, we combine our E2E and cascaded ST systems with Minimum Bayes-Risk decoding. Our com- bined system achieves a BLEU score of 21.6 and 19.1 on test2 and test3, respectively. 1 Introduction The performance of machine translation systems is closely tied to the amount of available training data. Regional dialects, which are less prevalent and pri- marily spoken languages, pose a challenge for these systems due to the scarcity of digital data, the ab- sence of standard orthography, and prevalence of non-standard grammar. The IWSLT 2023 dialect and low-resource track focuses these challenges. In this paper we present the JHU Tunisian Arabic to English speech translation systems submitted to the IWSLT 2023 dialectal and low-resource track (Agarwal et al., 2023). Arabic and its dialects form a dialect continuum anchored by Modern Standard Arabic (MSA) (Badawi et al., 2013). While MSA is the language of formal and written communi- cation, most native Arabic speakers colloquially use local dialects, which often lack a standardized written form. In many North African Arabic di- alects, including Tunisian, there is a significant code-switching with and borrowing from several contact languages: Berber and Romance languages like French, Spanish and Italian. Recent successes in machine translation (MT) of text for low-resource languages or non-standard dialects have entailed the use of large pretrained models such as mBART (Liu et al., 2020a) and NLLB (NLLB Team et al., 2022). These models have demonstrated state-of-the-art performance via transfer learning from higher-resource languages, particularly through related languages. However, there is a lack of understanding regarding how to ef- fectively integrate these models with speech recog- nition systems to develop speech translation sys- tems. To fill this gap we investigate dialect transfer by integrating large pretrained models with speech recognition models in end-to-end (E2E) and cas- caded speech translation (ST) systems. The key components of our system are: • Dialectal transfer from large pre-trained mod- els to improve translation in both E2E and Cascaded ST settings (§3.1,§3.2). • Improved ASR of dialectal speech by reduc- ing …

Content truncated to 500 words