Data Augmentation for End-to-End Speech Translation: FBK@IWSLT ‘19

Data Augmentation for End-to-End Speech Translation: FBK@IWSLT ’19 Mattia A. Di Gangi1,2, Matteo Negri1, Viet Nhat Nguyen2∗, Amirhossein Tebbifakhr1,2, Marco Turchi1 1Fondazione Bruno Kessler, Trento, Italy 2University of Trento, Italy {digangi|negri|vnguyen|atebbifakhr|turchi}@fbk.eu Abstract This paper describes FBK’s submission to the end-to-end speech translation (ST) task at IWSLT 2019. The task con- sists in the “direct” translation (i.e. without intermediate discrete representation) of English speech data derived from TED Talks or lectures into German texts. Our participation had a twofold goal: i) testing our latest models, and ii) eval- uating the contribution to model training of different data augmentation techniques. On the model side, we deployed our recently proposed S-Transformer with logarithmic dis- tance penalty, an ST-oriented adaptation of the Transformer architecture widely used in machine translation (MT). On the training side, we focused on data augmentation techniques recently proposed for ST and automatic speech recognition (ASR). In particular, we exploited augmented data in dif- ferent ways and at different stages of the process. We first trained an end-to-end ASR system and used the weights of its encoder to initialize the decoder of our ST model (trans- fer learning). Then, we used an English-German MT sys- tem trained on large data to translate the English side of the English-French training set into German, and used this newly-created data as additional training material. Finally, we trained our models using SpecAugment, an augmentation technique that randomly masks portions of the spectrograms in order to make them different at every training epoch. Our synthetic corpus and SpecAugment resulted in an improve- ment of 5 BLEU points over our baseline model on the test set of MuST-C En-De, reaching the score of 22.3 with a sin- gle end-to-end system. 1. Introduction End-to-end speech-to-text translation [1, 2] is the task of di- rectly translating a speech presented as an audio signal in a source language into a text in a different target language. The main goals of translating from the audio without inter- mediate discrete representations are to reduce translation la- tency and to avoid the cumulative effect of error propaga- tion. Cumulative errors typically affect the so-called cascade approaches to ST, which rely on a two-step processing in- volving separate speech recognition and translation compo- nents. Another advantage of the direct approach is the pos- ∗Work done during a summer internship at FBK sibility to translate languages without a formal script [3, 4], in particular for emergency situations. However, the high expectations so far have not been matched by the empirical results, which are generally much worse with an end-to-end system than with a cascade approach [5]. First, the main dif- ficulty of this approach comes from the scarce availability of annotated corpora for the end-to-end task. These corpora should come in the form of large collections of (src audio, tgt text) pairs, which are costly to acquire and still available only in small amount compared to the wealth of data exist- ing for the two related tasks of automatic speech recognition (ASR) and machine translation …

Content truncated to 500 words