Fine-tuning on Clean Data for End-to-End Speech Translation: FBK @ IWSLT 2018

Fine-tuning on Clean Data for End-to-End Speech Translation: FBK @ IWSLT 2018 Mattia Antonino Di Gangi1,2, Roberto Dess`ı 1*, Roldano Cattoni2, Matteo Negri2, Marco Turchi2 1University of Trento, Italy 2Fondazione Bruno Kessler, Italy 1roberto.dessi@unitn.it 2{digangi,cattoni,negri,turchi}@fbk.eu Abstract This paper describes FBK’s submission to the end-to-end English-German speech translation task at IWSLT 2018. Our system relies on a state-of-the-art model based on LSTMs and CNNs, where the CNNs are used to reduce the tem- poral dimension of the audio input, which is in general much higher than machine translation input. Our model was trained only on the audio-to-text parallel data released for the task, and fine-tuned on cleaned subsets of the original training corpus. The addition of weight normalization and label smoothing improved the baseline system by 1.0 BLEU point on our validation set. The final submission also fea- tured checkpoint averaging within a training run and ensem- ble decoding of models trained during multiple runs. On test data, our best single model obtained a BLEU score of 9.7, while the ensemble obtained a BLEU score of 10.24. 1. Introduction End-to-end speech translation (that is, the direct translation of an audio signal without intermediate transcription steps) has recently gained increasing interest in the scientific com- munity thanks to the recent advances of neural approaches in the related ASR and MT fields [1, 2, 3, 4, 5]. Effective ap- proaches to the task can become a useful solution to deal with languages that do not have a formal writing system [6], as it is possible to create a collection of spoken utterances with their respective translations in a more common language. We can also expect that, in the future, end-to-end speech translation systems will overcome problems related to the cumulative effect of speech recognition errors introduced in pipelined architectures. FBK’s submission to the IWSLT 2018 Speech Translation (ST) task relies on a single model that takes as input features extracted from an English audio signal and re- turns as output a written translation in German. As the in- put is not in raw wave form, one might argue that the “end- to-end” denomination does not fit in this formulation of the task. Nevertheless, since feeding the network with the input * Work performed during an internship at FBK features released by the task organizers was allowed, we ad- here to the looser definition of “end-to-end” implicit in this year’s task formulation.1 Our system was trained using the state-of-the-art sequence-to-sequence model based on LSTMs and CNNs introduced in [2]. Considering the high number of exper- iments to run, and the high number of epochs needed to train a speech translation model (up to 87 in the case of our final submission), the model was implemented using the fairseq2 [7] sequence-to-sequence learning toolkit from Facebook AI Research. The tool, which is tailored to NMT, was adapted to the ST task showing considerable reductions in training time compared to the same models implemented on other platforms (from hours to minutes in the processing of the same …

Content truncated to 500 words