Towards Efficient Simultaneous Speech Translation: CUNI-KIT System for Simultaneous Track at IWSLT 2023

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 389–396 July 13-14, 2023 c⃝2023 Association for Computational Linguistics Towards Efficient Simultaneous Speech Translation: CUNI-KIT System for Simultaneous Track at IWSLT 2023 Peter Polák1 and Danni Liu2 and Ngoc-Quan Ngoc2 Jan Niehues2 and Alexander Waibel2,3 and Ondˇrej Bojar1 polak@ufal.mff.cuni.cz 1 Charles University 2 Karlsruhe Institute of Technology 3 Carnegie Mellon University Abstract In this paper, we describe our submission to the Simultaneous Track at IWSLT 2023. This year, we continue with the successful setup from the last year, however, we adopt the latest meth- ods that further improve the translation quality. Additionally, we propose a novel online policy for attentional encoder-decoder models. The policy prevents the model to generate transla- tion beyond the current speech input by using an auxiliary CTC output layer. We show that the proposed simultaneous policy can be ap- plied to both streaming blockwise models and offline encoder-decoder models. We observe significant improvements in quality (up to 1.1 BLEU) and the computational footprint (up to 45 % relative RTF). 1 Introduction Simultaneous speech translation (SST) is the task of translating speech into text in a different lan- guage before the utterance is finished. The goal of SST is to produce a high-quality translation in real-time while maintaining low latency. However, these two objectives are conflicting. If we decrease the latency, the translation quality also drops. Last year’s IWSLT evaluation campaign (Anastasopou- los et al., 2022) showed that current methods for simultaneous speech translation can approach the translation quality of human interpreters (Polák et al., 2022). The disadvantage is a higher com- putation footprint that might make a widespread application prohibitive. This paper describes the CUNI-KIT submission to the Simultaneous translation track at IWSLT 2023 (Agarwal et al., 2023). Following our last year’s submission (Polák et al., 2022), we continue in our effort to onlinize the robust offline speech translation models. However, the main goal of this submission is to improve the computational foot- print. To this end, we propose a novel online policy based on CTC. As we experimentally document, the online CTC policy can be used to onlinize the offline models achieving a 45 % improvement in real time factor (RTF) as well as to improve the quality of the streaming blockwise models (Tsunoo et al., 2021). Aside from improving the online policy, we also adopt the novel improved stream- ing beam search (Polák et al., 2023) that further improves the translation quality. Our contributions are as follows: • We adopt the latest online decoding algorithm that improves the translation quality of robust offline models in the simultaneous regime, • We propose a novel online policy that signifi- cantly – lowers the computational complexity of the online decoding with robust offline models while maintaining the same or only slightly worse translation quality, – improves the translation quality of the streaming blockwise models while main- taining the same latency, • We demonstrate that our systems can run on hardware accessible to a wide …

Content truncated to 500 words