Effective combination of pretrained models - KIT@IWSLT2022

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 190 - 197 May 26-27, 2022 c⃝2022 Association for Computational Linguistics Effective combination of pretrained models - KIT@IWSLT2022 Ngoc-Quan Pham1 and Tuan-Nam Nguyen1 and Thai-Binh Nguyen1 and Danni Liu2 and Carlos Mullov1 and Jan Niehues1 and Alexander Waibel1,3 1Karlsruhe Institute of Technology 2Maastricht University 3Carnegie Mellon University, Pittsburgh PA, USA firstname.lastname@kit.edu firstname.lastname@maastrichtuniversity.nl Abstract Pretrained models in acoustic and textual modalities can potentially improve speech translation for both Cascade and End-to-end approaches. In this evaluation, we aim at em- pirically looking for the answer by using the wav2vec, mBART50 and DeltaLM models to improve text and speech translation models. The experiments showed that the presence of these models together with an advanced au- dio segmentation method results in an improve- ment over the previous End-to-end system by up to 7 BLEU points. More importantly, the ex- periments showed that given enough data and modeling capacity to overcome the training dif- ficulty, we can outperform even very competi- tive Cascade systems. In our experiments, this gap can be as large as 2.0 BLEU points, the same gap that the Cascade often led over the years. 1 Introduction Speech translation (ST) has been the main theme of IWSLT for more than a decade and it goes without saying between the traditional Cascade approach and the recent End-to-end (E2E) possibility, the for- mer has always been preferred. Being able to divide the complicated ST to smaller sub-problems: au- tomatic recognition, (often) re-segmentation (Cho et al., 2017) and machine translation, the cascade approach has the advantage of using more data to separately optimize the components. The E2E, on the other hand, relies on a single network archi- tecture that requires an explicit speech-translation dataset. Over the years of participation, we observed that the performance gap between E2E and cascade is reduced (Anastasopoulos et al., 2021), and there are three negative factors that outweigh the advantages of having a single architecture without the problem of error propagation (Sperber and Paulik, 2020). • Data utilization: the end2end model can only be directly trained on parallel speech transla- tion data, which is often lacking compared to speech-transcription or text translation data. Previously the SLT models would require a necessary pre-training step with ASR in order to have comparable results with cas- cade (Bansal et al., 2018; Pham et al., 2020c). • Modeling power. The transition from shallow LSTM-based models (Sperber et al., 2019) to Transformer-based models (Pham et al., 2020a) resulted in a big leap in model capacity and showed the potential of the E2E approach. • Better audio segmentation. Decoding directly from long audio files is infeasible due to the expensive memory requirement and the presence of other distractions such as breaks, noise or music. Applying either cascade or E2E models absolutely requires an audio seg- mentation step performed by a voice activ- ity detection system. While the cascade sys- tems can handle imprecise cuts based on a re-segmentation process (Cho et al., 2017), …

Content truncated to 500 words