GMU Systems for the IWSLT 2023 Dialect and Low-resource Speech Translation Tasks

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 269–276 July 13-14, 2023 c⃝2023 Association for Computational Linguistics GMU Systems for the IWSLT 2023 Dialect and Low-resource Speech Translation Tasks Jonathan Kabala Mbuya George Mason University jmbuya@gmu.edu Antonios Anastasopoulos George Mason University antonis@gmu.edu Abstract This paper describes the GMU Systems for the IWSLT 2023 Dialect and Low-resource Speech Translation Tasks. We submitted systems for five low-resource tasks and the dialectal task. In this work, we explored self-supervised pre- trained speech models and finetuned them on speech translation downstream tasks. We use the Wav2vec 2.0, XLSR-53, and Hubert as self- supervised models. Unlike Hubert, Wav2vec 2.0 and XLSR-53 achieve the best results when we remove the top three layers. Our results show that Wav2vec 2.0 and Hubert perform similarly with their relative best configuration. In addition, we found that Wav2vec 2.0 pre- trained on audio data of the same language as the source language of a speech translation model achieves better results. For the low- resource setting, the best results are achieved using either the Wav2vec 2.0 or Hubert models, while XLSR-53 achieves the best results for the dialectal transfer task. We find that XLSR- 53 does not perform well for low-resource tasks. Using Wav2vec 2.0, we report close to 2 BLEU point improvements on the test set for the Tamasheq-French compared to the baseline system at the IWSLT 2022. 1 Introduction Recently, speech-to-text translation (S2T) has re- ceived a lot of focus in the community where neu- ral, end-to-end approaches outperform traditional statistical approaches (Weiss et al., 2017). Recent neural approaches to S2T have shown superior per- formance on this task (Fang et al., 2022; Tang et al., 2022). Despite the success of neural approaches to S2T, data scarcity is one of the significant chal- lenges, given that neural networks require hundreds to thousands of hours of labeled data to train a good speech translation model (Sperber and Paulik, 2020). This makes developing such S2T models challenging, especially for low-resource languages. The IWSLT 2023 Low-resource and dialectal shared tasks (Agarwal et al., 2023) give the possi- bilities for researchers to find innovative ways to develop speech translation systems for languages with limited data. Unlike previous years, this year noticed an addition of more low-resource languages language pairs (up to 6) in addition to a dialect lan- guage pair. This paper describes the GMU submissions to the low-resource and dialectal tasks. Our systems use self-supervised pre-trained speech models to improve speech translation models’ performance in general, particularly for low-resource languages. Self-supervised pre-training is possible because unlabeled data (i.e., audio or text) can be obtained easier compared to labeled data. Previous research has addressed using self-supervised speech models for speech translation (Wu et al., 2020; Nguyen et al., 2020; Popuri et al., 2022). However, these prior work did not consider exploring the impact of different layers of these self-supervised models to maximize the performance of S2T models. In this paper, we consider three self-supervised speech models: Wav2vec 2.0 …

Content truncated to 500 words