End-to-End Speech-Translation with Knowledge Distillation: FBK@IWSLT2020

Proceedings of the 17th International Conference on Spoken Language Translation (IWSLT), pages 80–88 July 9-10, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 80 End-to-End Speech-Translation with Knowledge Distillation: FBK@IWSLT2020 Marco Gaido1,2, Mattia Antonino Di Gangi1,2, Matteo Negri1, Marco Turchi1 1Fondazione Bruno Kessler, Trento, Italy 2University of Trento, Italy {mgaido|digangi|negri|turchi}@fbk.eu Abstract This paper describes FBK’s participation in the IWSLT 2020 offline speech translation (ST) task. The task evaluates systems’ ability to translate English TED talks audio into Ger- man texts. The test talks are provided in two versions: one contains the data already seg- mented with automatic tools and the other is the raw data without any segmentation. Par- ticipants can decide whether to work on cus- tom segmentation or not. We used the pro- vided segmentation. Our system is an end- to-end model based on an adaptation of the Transformer for speech data. Its training pro- cess is the main focus of this paper and it is based on: i) transfer learning (ASR pretraining and knowledge distillation), ii) data augmenta- tion (SpecAugment, time stretch and synthetic data), iii) combining synthetic and real data marked as different domains, and iv) multi- task learning using the CTC loss. Finally, af- ter the training with word-level knowledge dis- tillation is complete, our ST models are fine- tuned using label smoothed cross entropy. Our best model scored 29 BLEU on the MuST-C En-De test set, which is an excellent result compared to recent papers, and 23.7 BLEU on the same data segmented with VAD, showing the need for researching solutions addressing this specific data condition. 1 Introduction The offline speech translation task consists in gen- erating the text translation of speech audio record- ings into a different language. In particular, the IWSLT2020 task (Ansari et al., 2020) evaluates German translation of English recordings extracted from TED talks. The test dataset is provided to participants both segmented in a sentence-like for- mat using a Voice Activity Detector (VAD) and in the original unsegmented form. Although cus- tom segmentation of the data can provide drastic improvements in the final scores, in our work we have not addressed it, participating only with the provided segmentation. Two main approaches are possible to face the speech translation task. The classic one is the cas- cade solution, which includes automatic speech recognition (ASR) and machine translation (MT) components. The other option is an end-to-end (E2E) solution, which performs ST with a single sequence-to-sequence model. Both of them are al- lowed for the IWSLT2020 task, but our submission is based on an E2E model. E2E ST models gained popularity in the last few years. Their rise is due to the lack of error propaga- tion and the reduced latency in generating the out- put compared to the traditional cascaded approach. Despite these appealing properties, they failed so far to reach the same results obtained by cascade systems, as shown also by last year’s IWSLT cam- paign (Niehues et al., 2019). One reason for this is the limited amount of parallel corpora …

Content truncated to 500 words