On Using SpecAugment for End-to-End Speech Translation

On Using SpecAugment for End-to-End Speech Translation Parnia Bahar1,2, Albert Zeyer1,2, Ralf Schl¨uter1 and Hermann Ney1,2 1Human Language Technology and Pattern Recognition Group Computer Science Department, RWTH Aachen University, 52062 Aachen, Germany 2AppTek, 52062 Aachen, Germany {bahar, zeyer, schlueter, ney}@cs.rwth-aachen.de Abstract This work investigates a simple data augmentation technique, SpecAugment, for end-to-end speech translation. SpecAug- ment is a low-cost implementation method applied directly to the audio input features and it consists of masking blocks of frequency channels, and/or time steps. We apply SpecAug- ment on end-to-end speech translation tasks and achieve up to +2.2% BLEU on LibriSpeech Audiobooks En→Fr and +1.2% on IWSLT TED-talks En→De by alleviating over- fitting to some extent. We also examine the effectiveness of the method in a variety of data scenarios and show that the method also leads to significant improvements in various data conditions irrespective of the amount of training data. 1. Introduction Traditional speech-to-text translation (ST) systems have been build in a cascaded fashion comprised of an automatic speech recognition (ASR) model trained on paired speech- transcribed data and a machine translation (MT) model trained on bilingual text data. Recent advancements in both ASR [1–6] and MT [7–11] have inspired the end-to-end di- rect ST models which can be trained using a translation speech corpus [12, 13]. Some appealing advantages of the direct models are: (1) no error accumulation from the rec- ognizer, (2) faster decoding throughput and (3) less compu- tational power in total by training all parameters jointly. In spite of these properties, training such end-to-end ST models requires a moderate amount of paired translated speech-to- text data which is not easy to acquire. Therefore these mod- els tend to overfit easily. In the absence of an adequate volume of training data, one remedy is generating synthetic data like back-translation (BT) [14] as the most common data augmentation method to leverage monolingual data. The idea is to use a pre- trained model to convert weakly supervised data into speech- to-translation pairs for ST training [15]. One way is to use a pre-trained source-to-target MT model to translate ASR tran- scription into the target language. Another method is the use of a pre-trained text-to-speech (TTS) model to generate speech data from a monolingual text. However, these meth- ods require some effort to train an additional model, as well as computational power to generate a moderate amount of (noisy) synthetic data, which in some cases can be too ex- pensive to be obtained. Another method is data augmentation by which new syn- thetic training samples are generated by corrupting the ini- tial audio data and conserving the same label as the origi- nal training sample. Audio-level speech augmentation can be done in different ways such as noise injection (adding random noise), shifting time (transmitting time series for- ward/backward with a few seconds), speed perturbation (ex- panding time series by a speed rate) and changing the fre- quency pitch randomly. Besides increasing the quantity of training data, data augmentation often make the model in- …

Content truncated to 500 words