The BIGAI Offline Speech Translation Systems for IWSLT 2023 Evaluation

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 123–129 July 13-14, 2023 c⃝2023 Association for Computational Linguistics The BIGAI Offline Speech Translation Systems for IWSLT 2023 Evaluation Zhihang Xie Beijing Institute of General Artificial Intelligence zhihangxie@gmail.com Abstract This paper describes the BIGAI’s submission to IWSLT 2023 Offline Speech Translation task on three language tracks from English to Chi- nese, German and Japanese. The end-to-end systems are built upon a Wav2Vec2 model for speech recognition and mBART50 models for machine translation. An adapter module is ap- plied to bridge the speech module and the trans- lation module. The CTC loss between speech features and source token sequence is incorpo- rated during training. Experiments show that the systems can generate reasonable transla- tions on three languages. The proposed models achieve BLEU scores of 22.3, 10.7 and 33.0 on tst2023 en→de, en→ja and en→zh TED datasets. It is found that the performance is decreased by a significant margin on complex scenarios like presentations and interviews. 1 Introduction Speech translation aims to solve the problem of translating speech waveform in source language into written text in target language. Cascade sys- tems decompose the problem into automatic speech recognition (ASR) to transcribe source speech into source text and machine translation (MT) to trans- late source text into target text (Wang et al., 2021b; Zhang et al., 2022a). It is clear that such architec- ture has the advantage of ensembling results from state-of-the-art (SOTA) ASR models and MT mod- els and the disadvantages of accumulating subsys- tem errors and discarding paralinguistic features. Recent end-to-end speech translation (E2E ST) sys- tems have shown the potential to outperform cas- cade systems (Hrinchuk et al., 2022; Shanbhogue et al., 2022). However, due to the lack of high- quality parallel training data, it is difficult to quan- tify the gap between the two categories. Inspired by Zhang et al.’s (2022b) work, this submission explores various techniques to address problems in speech translation. 1) Perform fine- grained data filtering by calculating WERs for speech data and alignment scores for translation data. 2) Apply a straightforward split-and-merge method to split long audio clips into short seg- ments. 3) Employ a three-stage training strategy to concatenate the finetuned speech module and the translation module. 4) Incorporate connectionist temporal classification (CTC) loss to leverage the divergence between speech features and source to- ken sequences (Graves et al., 2006). Experiments are carried out to perform speech translation at sen- tence level and corpus level. The performance of the three PT36 models is finally evaluated on the tst2023 datasets with automatic metrics. The rest of this paper is organized as follows. Section 2 describes how speech data and translation data are processed in the experiments. Section 3 explains how finetuned models are assembled to perform speech translation on all three languages. Section 4 illustrates experiment setups, results and analysis. Section 5 concludes the submission. 2 Data Processing 2.1 Speech Corpora Under the constrained condition, there are five speech datasets used to …

Content truncated to 500 words