CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 277 - 285 May 26-27, 2022 c⃝2022 Association for Computational Linguistics CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022 Peter Polák1 and Ngoc-Quan Ngoc2 and Tuan-Nam Nguyen2 and Danni Liu3 Carlos Mullov2 and Jan Niehues2 and Ondˇrej Bojar1 and Alexander Waibel2,4 polak@ufal.mff.cuni.cz 1 Charles University 2 Karlsruhe Institute of Technology 3 Maastricht University 4 Carnegie Mellon University Abstract In this paper, we describe our submission to the Simultaneous Speech Translation at IWSLT 2022. We explore strategies to utilize an of- fline model in a simultaneous setting without the need to modify the original model. In our experiments, we show that our onlinization al- gorithm is almost on par with the offline setting while being 3× faster than offline in terms of latency on the test set. We also show that the onlinized offline model outperforms the best IWSLT2021 simultaneous system in medium and high latency regimes and is almost on par in the low latency regime. We make our system publicly available.1 1 Introduction This paper describes the CUNI-KIT submission to the Simultaneous Speech Translation task at IWSLT 2022 (Anastasopoulos et al., 2022) by Charles University (CUNI) and Karlsruhe Institute of Technology (KIT). Recent work on end-to-end (E2E) simultaneous speech-to-text translation (ST) is focused on train- ing specialized models specifically for this task. The disadvantage is the need of storing an extra model, usually a more difficult training and infer- ence setup, increased computational complexity (Han et al., 2020; Liu et al., 2021) and risk of per- formance degradation if used in offline setting (Liu et al., 2020a). In this work, we base our system on a robust mul- tilingual offline ST model that leverages pretrained wav2vec 2.0 (Baevski et al., 2020) and mBART (Liu et al., 2020b). We revise the onlinization ap- proach by Liu et al. (2020a) and propose an im- proved technique with a fully controllable quality- latency trade-off. We demonstrate that without any change to the offline model, our simultaneous sys- tem in the mid- and high-latency regimes is on par 1https://hub.docker.com/repository/ docker/polape7/cuni-kit-simultaneous with the offline performance. At the same time, the model outperforms previous IWSLT systems in medium and high latency regimes and is almost on par in the low latency regime. Finally, we observe a problematic behavior of the average lagging met- ric for speech translation (Ma et al., 2020) when dealing with long hypotheses, resulting in negative values. We propose a minor change to the metric formula to prevent this behavior. Our contribution is as follows: • We revise and generalize onlinization pro- posed by Liu et al. (2020a); Nguyen et al. (2021) and discover parameter enabling quality-latency trade-off, • We demonstrate that one multilingual offline model can serve as simultaneous ST for three language pairs, • We demonstrate that an improvement in the offline model leads also to an improvement in the online regime, • We propose a change to the average lagging metric that avoids negative …

Content truncated to 500 words