ON-TRAC Consortium Systems for the IWSLT 2023 Dialectal and Low-resource Speech Translation Tasks

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 219–226 July 13-14, 2023 c⃝2023 Association for Computational Linguistics ON-TRAC consortium systems for the IWSLT 2023 dialectal and low-resource speech translation tasks Antoine Laurent1, Souhir Gahbiche5, Ha Nguyen2, Haroun Elleuch4, Fethi Bougares4, Antoine Thiol5, Hugo Riguidel1,3, Salima Mdhaffar2, Gaëlle Laperrière2, Lucas Maison2, Sameer Khurana6, Yannick Estève2 1LIUM - Le Mans University, France, 2LIA - Avignon University, France, 3Systran - France, 4ELYADATA - Tunis, Tunisia, 5Airbus - France, 6MIT Computer Science and Artificial Intelligence Laboratory, Cambridge, MA, USA Abstract This paper describes the ON-TRAC consor- tium speech translation systems developed for IWSLT 2023 evaluation campaign. Overall, we participated in three speech translation tracks featured in the low-resource and dialect speech translation shared tasks, namely; i) spoken Tamasheq to written French, ii) spoken Pashto to written French, and iii) spoken Tunisian to written English. All our primary submissions are based on the end-to-end speech-to-text neu- ral architecture using a pre-trained SAMU- XLSR model as a speech encoder and an mbart model as a decoder. The SAMU-XLSR model is built from the XLS-R 128 in order to gen- erate language agnostic sentence-level embed- dings. This building is driven by the LaBSE model trained on a multilingual text dataset. This architecture allows us to improve the input speech representations and achieve significant improvements compared to conventional end- to-end speech translation systems. 1 Introduction IWSLT is a unique opportunity that allows each year the assessment of progress made in the area of Spoken Language Translation (SLT). This as- sessment is made possible throughout the organ- isation of an evaluation campaign including var- ious shared tasks that address specific scientific challenges of the SLT domain. In addition to the well-established shared tasks, IWSLT organisers introduce new tasks to address the many challenges settings related to SLT area like data scarcity, mul- tilingualism, time and computation constraints, etc. In this context, the IWSLT 2023 proposes two interesting shared tasks: low-resource and dialect speech translation (ST). The former aims to as- sess the exploitability of current translation sys- tems in data scarcity settings. The latter focuses on the assessment of the systems’ capabilities in noisy settings: different dialects are mixed in a single dataset of spontaneous speech. For the low-resource task, several language pairs were proposed this year. In this paper, we focus on Tamasheq-French, Tunisian Arabic-English and Pashto-French. This paper reports the ON-TRAC consortium submissions for the aforementioned tasks. The ON-TRAC Consortium is composed of researchers from three academic laboratories, LIUM (Le Mans University - France), LIA (Avignon University - France), MIT (Cambridge - USA) together with three industrial partners: Airbus France, ELYA- DATA and Systran. Our systems for the dialect task focus on both cascaded and end-to-end approaches for ST. For the low-resource task, we focus on the leveraging of models based on self-supervised learning (SSL), and on the training of ST models with joint automatic speech recognition (ASR), ma- chine translation (MT) and ST losses. This paper is organized as follows. Section …

Content truncated to 500 words