IMS’ Systems for the IWSLT 2021 Low-Resource Speech Translation Task

Proceedings of the 18th International Conference on Spoken Language Translation, pages 175–181 Bangkok, Thailand (Online), August 5–6, 2021. ©2021 Association for Computational Linguistics 175 IMS’ Systems for the IWSLT 2021 Low-Resource Speech Translation Task Pavel Denisov, Manuel Mager, Ngoc Thang Vu Institute for Natural Language Processing, University of Stuttgart {pavel.denisov,manuel.mager,thangvu}@ims.uni-stuttgart.de Abstract This paper describes the submission to the IWSLT 2021 Low-Resource Speech Transla- tion Shared Task by IMS team. We utilize state- of-the-art models combined with several data augmentation, multi-task and transfer learning approaches for the automatic speech recogni- tion (ASR) and machine translation (MT) steps of our cascaded system. Moreover, we also ex- plore the feasibility of a full end-to-end speech translation (ST) model in the case of very con- strained amount of ground truth labeled data. Our best system achieves the best performance among all submitted systems for Congolese Swahili to English and French with BLEU scores 7.7 and 13.7 respectively, and the second best result for Coastal Swahili to English with BLEU score 14.9. 1 Introduction We participate in the low-resource speech transla- tion task of IWSLT 2021. This task is organized for the first time, and it focuses on three speech translation directions this year: Coastal Swahili to English (swa→eng), Congolese Swahili to French (swc→fra) and Congolese Swahili to En- glish (swc→eng). Working on under-represented and low-resource languages is of special relevance for the inclusion into technologies of big parts of the world population. The Masakhane initiative (Nekoto et al., 2020) has opened the doors for large scale participatory research on languages of the African continent, to which Swahili belongs to. Our Speech-to-Text translation systems aim to con- tribute to this global effort. A common problem for these languages is the small amount of data. This is also true for the lan- guage pairs of the shared task: the provided data contains a small amount of translated speech sam- ples for each pair, but the participants are allowed to use additional data and pre-trained models for the sub-tasks of ASR and MT. We utilize most of the suggested additional data resources to train and tune sequence-to-sequence ASR and MT compo- nents. Our primary submission is the cascaded system built of Conformer end-to-end ASR model and Transformer MT model. Our contrastive sys- tem is end-to-end ST system utilizing parameters transfer from the Encoder part of ASR model and the full MT model. Both ASR and MT components of the cascaded system initially yield good results on their own, but the discrepancy between language formats (spoken vs. written) in ASR and MT corpora causes degra- dation by 47% in resulting scores. To adapt the MT system to the output of the ASR, we transform the Swahili source data to output similar to one of an ASR system. To further increase the perfor- mance of our MT system, we leverage both source formats (original Swahili text and simulated ASR output Swahili) into a multi-task framework. This approach improves our results by 17%, mostly for the English target language. Our …

Content truncated to 500 words