Amazon Alexa AI’s System for IWSLT 2022 Offline Speech Translation Shared Task

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 169 - 176 May 26-27, 2022 c⃝2022 Association for Computational Linguistics Amazon Alexa AI’s System for IWSLT 2022 Offline Speech Translation Shared Task Akshaya Vishnu Kudlu Shanbhogue∗ Ran Xue∗ Ching-Yun Chang Sarah Campbell Amazon Alexa AI {ashanbho,ranxue,cychang,srh}@amazon.com Abstract This paper describes Amazon Alexa AI’s sub- mission to the IWSLT 2022 Offline Speech Translation Task. Our system is an end-to- end speech translation model that leverages pretrained models and cross modality transfer learning. We detail two improvements to the knowledge transfer schema. First, we imple- mented a new loss function that reduces knowl- edge gap between audio and text modalities in translation task effectively. Second, we in- vestigate multiple finetuning strategies includ- ing sampling loss, language grouping and do- main adaption. These strategies aims to bridge the gaps between speech and text translation tasks. We also implement a multi-stage seg- mentation and merging strategy that yields im- provements on the unsegmented development datasets. Results show that the proposed loss function consistently improves BLEU scores on the development datasets for both English- German and multilingual models. Addition- ally, certain language pairs see BLEU score improvements with specific finetuning strate- gies. 1 Introduction Multilingual Spoken Language Translation (SLT) enables translation of audio into text in multiple languages. Traditionally, SLT is solved by cas- cading automatic speech recognition (ASR) mod- els, which convert audio to transcribed text, with text-to-text translation models. End-to-end (E2E) models, such as FAIR Speech Translation System (Tang et al., 2021a), allow a single model to trans- late from speech to text. Recent advances in E2E models show comparable results with cascaded ar- chitectures (Anastasopoulos et al., 2021; Ansari et al., 2020). Our baseline end-to-end speech translation sys- tem leverages large-scale pretrained models on dif- ∗Akshaya Vishnu Kudlu Shanbhogue and Ran Xue have equal contribution to this work. ferent data modalities following the approach pro- posed by Tang et al. (2021a). We adopt dynamic dual skew divergence (DDSD) loss function (Li et al., 2021b) to replace cross entropy (CE) for effective knowledge transfer from pretrained text- to-text (T2T) translation model to speech-to-text (S2T) translation model through joint task training. We observe that DDSD consistently outperforms CE across all language directions. Our multilingual model supports translation of English (en) audio to German (de), Japanese (ja) and Chinese (zh). We find that finetuning this model based on language groups can improve the performance of the model. Additionally, we find that finetuning models by considering alternate translations can lead to subtle improvements in the overall performance of the models. While work- ing with unsegmented data, we show that using a custom audio segmentation strategy can improve the translation performance by around +2.0 BLEU points. On IWSLT 2022 blind test sets, our sys- tem achieves 22.6, 15.3, and 30.4 BLEU score for en→de, en→ja, and en→zh respectively. On the progression test set, our E2E speech translation sys- tem performs on par with IWSLT 2021 winning cascaded system (Anastasopoulos et al., 2021). 2 …

Content truncated to 500 words