The USTC’s Dialect Speech Translation System for IWSLT 2023

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 102–112 July 13-14, 2023 c⃝2023 Association for Computational Linguistics The USTC’s Dialect Speech Translation System for IWSLT 2023 Pan Deng1 Shihao Chen1 Weitai Zhang1,2 Jie Zhang1 Lirong Dai1 1University of Science and Technology of China, Hefei, China 2iFlytek Research, Hefei, China {pdeng, shchen16, zwt2021}@mail.ustc.edu.cn; {jzhang6, lrdai}@ustc.edu.cn Abstract This paper presents the USTC system for the IWSLT 2023 Dialectal and Low-resource shared task, which involves translation from Tunisian Arabic to English. We aim to investi- gate the mutual transfer between Tunisian Ara- bic and Modern Standard Arabic (MSA) to en- hance the performance of speech translation (ST) by following standard pre-training and fine-tuning pipelines. We synthesize a substan- tial amount of pseudo Tunisian-English paired data using a multi-step pre-training approach. Integrating a Tunisian-MSA translation mod- ule into the end-to-end ST model enables the transfer from Tunisian to MSA and facilitates linguistic normalization of the dialect. To in- crease the robustness of the ST system, we op- timize the model’s ability to adapt to ASR er- rors and propose a model ensemble method. Results indicate that applying the dialect trans- fer method can increase the BLEU score of dialectal ST. It is shown that the optimal sys- tem ensembles both cascaded and end-to-end ST models, achieving BLEU improvements of 2.4 and 2.8 in test1 and test2 sets, respectively, compared to the best published system. 1 Introduction In this paper, we present the USTC’s submission to the Dialectal and Low-resource track of IWSLT 2023 Evaluation Campaign (Agarwal et al., 2023), aiming to translate Tunisian Arabic speech to En- glish text. Modern Standard Arabic (MSA) is the official language of Arabic-spoken countries. How- ever, Arabic dialects like Tunisian and Egyptian are prevalent in everyday communication, exhibiting a similar relation between Chinese and Cantonese. MSA benefits from an abundant supply of unla- beled speech and text data, as well as relatively adequate automatic speech recognition (ASR) and machine translation (MT) paired data. In contrast, dialectical forms of Arabic have much less paired data and more irregularities in both pronunciation and writing (Ben Abdallah et al., 2020). This paper aims to explore the transfer between high-resource MSA and low-resource Tunisian di- alects, as well as effective training and decoding strategies for speech translation (ST) tasks related to low-resource dialects. To facilitate dialect trans- fer, we introduce two approaches. Firstly, we pre- train a model using high-resource MSA data, which is then fine-tuned using low-resource Tunisian data. This approach involves transferring model parame- ters and can be used to train various models, e.g., ASR, MT, end-to-end ST. Secondly, we also de- velop two transformation models for explicit di- alect transfer. On one hand, for the augmentation of MT data, we build an MT model that translates MSA into Tunisian, resulting in a vast amount of pseudo Tunisian-English paired data. On the other hand, the Tunisian-MSA MT encoder module is built and then integrated into the end-to-end ST model, which can implicitly normalize dialectal expressions. …

Content truncated to 500 words