ON-TRAC’ systems for the IWSLT 2021 low-resource speech translation and multilingual speech translation shared tasks

Proceedings of the 18th International Conference on Spoken Language Translation, pages 169–174 Bangkok, Thailand (Online), August 5–6, 2021. ©2021 Association for Computational Linguistics 169 ON-TRAC’ systems for the IWSLT 2021 low-resource speech translation and multilingual speech translation shared tasks Hang Le2, Florentin Barbier4, Ha Nguyen1,2, Natalia Tomanshenko1, Salima Mdhaffar1, Souhir Gahbiche4, Fethi Bougares3, Benjamin Lecouteux2, Didier Schwab2, Yannick Est`eve1 ON-TRAC consortium (LIA1, LIG2, LIUM3, Airbus4 - France) Abstract This paper describes the ON-TRAC Con- sortium translation systems developed for two challenge tracks featured in the Evalua- tion Campaign of IWSLT 2021, low-resource speech translation and multilingual speech translation. The ON-TRAC Consortium is composed of researchers from three French academic laboratories and an industrial part- ner: LIA (Avignon Universit´e), LIG (Univer- sit´e Grenoble Alpes), LIUM (Le Mans Uni- versit´e), and researchers from Airbus. A pipeline approach was explored for the low- resource speech translation task, using a hy- brid HMM/TDNN automatic speech recogni- tion system fed by wav2vec features, cou- pled to an NMT system. For the multilingual speech translation task, we investigated the use of a dual-decoder Transformer that jointly tran- scribes and translates an input speech signal. This model was trained in order to translate from multiple source languages to multiple tar- get ones. 1 Introduction In the two last editions of the IWSLT evaluation campaigns, the ON-TRAC consortium focused on end-to-end offline speech translation and simul- taneous speech translation (Nguyen et al., 2019; Elbayad et al., 2020). In 2021, we chose to focus on low-resource speech translation and multilin- gual speech translation by using two different kinds of approaches: a cascaded speech-to-text transla- tion (combining source language automatic speech recognition (ASR) and source-to-target text trans- lation) to process the low resource speech transla- tion tasks, and a neural end-to-end model for the multilingual speech translation task. For the low resource task, we investigated the use of speech fea- tures extracted by a neural model pretrained by self supervision the wav2vec XLSR-53 model (Con- neau et al., 2020) in order to process Swahili lan- guages by a classical hybrid Markovian/neuronal ASR system. The ASR outputs were processed by neural machine text-to-text translation systems dedicated to the two targeted language pairs. For the multilingual speech translation task, we inves- tigated the use of a dual-decoder Transformer that jointly transcribes and translates an input speech. This model was trained in order to translate from multiple source languages to multiple target ones. The ON-TRAC Consortium is composed of re- searchers from three French academic laboratories and an industrial partner: LIA (Avignon Univer- sit´e), LIG (Universit´e Grenoble Alpes), LIUM (Le Mans Universit´e), and researchers from Airbus. 2 Low resource speech translation The task of the low resource speech transla- tion track was to build the speech transcrip- tion/translation system for transcribing and/or trans- lating between the two language pairs: • Coastal Swahili (swa) to English (eng) • Congolese Swahili (swc) to French (fra) 2.1 ASR system The same ASR models were used for both test datasets: Coastal Swahili (swa) and Congolese Swahili (swc). …

Content truncated to 500 words