The HW-TSC’s Simultaneous Speech-to-Text Translation System for IWSLT 2023 Evaluation

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 376–382 July 13-14, 2023 c⃝2023 Association for Computational Linguistics The HW-TSC’s Simultaneous Speech-to-Text Translation system for IWSLT 2023 evaluation Jiaxin GUO, Daimeng Wei, Zhanglin Wu, Zongyao Li, Zhiqiang Rao, Minghan Wang, Hengchao Shang, Xiaoyu Chen, Zhengzhe Yu, Shaojun Li, Yuhao Xie, Lizhi Lei, Hao Yang {guojiaxin1, weidaimeng, wuzhanglin2, lizongyao, raozhiqiang, wangminghan, shanghengchao, chenxiaoyu35, yuzhengzhe, lishaojun18, xieyuhao2, leilizhi, yanghao30}@huawei.com Abstract In this paper, we present our submission to the IWSLT 2023 (Agarwal et al., 2023) Simulta- neous Speech-to-Text Translation competition. Our participation involves three language direc- tions: English-German, English-Chinese, and English-Japanese. Our proposed solution is a cascaded incremental decoding system that comprises an ASR model and an MT model. The ASR model is based on the U2++ architec- ture and can handle both streaming and offline speech scenarios with ease. Meanwhile, the MT model adopts the Deep-Transformer archi- tecture. To improve performance, we explore methods to generate a confident partial target text output that guides the next MT incremen- tal decoding process. In our experiments, we demonstrate that our simultaneous strategies achieve low latency while maintaining a loss of no more than 2 BLEU points when compared to offline systems. 1 Introduction This paper describes the HW-TSC‘s submission to the Simultaneous Speech-to-Text Translation (SimulS2T) task at IWSLT 2023 (Agarwal et al., 2023). From a systems architecture perspective, current research on simultaneous speech-to-text translation (SimulS2T) can be categorized into two forms: cas- cade and end-to-end. Cascade systems typically consist of a streaming Automatic Speech Recogni- tion (ASR) module and a streaming text-to-text ma- chine translation (MT) module, with the possibil- ity of incorporating additional correction modules. While integrating these modules can be complex, training each module with sufficient data resources can prove to be worthwhile. Alternatively, an end- to-end approach is also an option for SimulS2T, where translations can be directly generated from a unified model with speech inputs. However, it is important to note that bilingual speech transla- tion datasets, which are necessary for end-to-end models, are still scarce resources. The current efforts in simultaneous speech-to- text translation (SimulS2T) concentrate on devel- oping dedicated models that are tailored to this specific task. However, this approach has certain drawbacks, such as the requirement of an additional model, which typically involves a more challenging training and inference process, as well as height- ened computational demands and the possibility of decreased performance when utilized in an offline environment. Our approach for this study involves utilizing a sturdy offline ASR model and a robust offline MT model as the foundation for our system. By modify- ing the onlinization approach of (Polák et al., 2022) and introducing an enhanced technique that can be seamlessly integrated into the cascade system, we are able to demonstrate that our simultaneous sys- tem can perform at the similar level as the offline models under strict latency restrictions without any adjustments to the original models. Furthermore, our system even surpasses previous higher latency IWSLT systems. Our contribution is …

Content truncated to 500 words