CTC-based Compression for Direct Speech Translation

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 690–696 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 690 CTC-based Compression for Direct Speech Translation Marco Gaido†, ∗, Mauro Cettolo†, Matteo Negri†, Marco Turchi† †Fondazione Bruno Kessler ∗University of Trento {mgaido,cettolo,negri,turchi}@fbk.eu Abstract Previous studies demonstrated that a dynamic phone-informed compression of the input au- dio is beneficial for speech translation (ST). However, they required a dedicated model for phone recognition and did not test this solution for direct ST, in which a single model trans- lates the input audio into the target language without intermediate representations. In this work, we propose the first method able to per- form a dynamic compression of the input in direct ST models. In particular, we exploit the Connectionist Temporal Classification (CTC) to compress the input sequence according to its phonetic characteristics. Our experiments demonstrate that our solution brings a 1.3- 1.5 BLEU improvement over a strong base- line on two language pairs (English-Italian and English-German), contextually reducing the memory footprint by more than 10%. 1 Introduction Speech translation (ST) is the process that converts utterances in one language into text in another lan- guage. Traditional approaches to ST consist of separate modules, each dedicated to an easier sub- task, which are eventually integrated in a so-called cascade architecture (Stentiford and Steer, 1988; Waibel et al., 1991). Usually, its main components are an automatic speech recognition (ASR) model - which generates the transcripts from the audio - and a machine translation (MT) model - which translates the transcripts into the target language. A newer approach is direct ST, in which a single model performs the whole task without interme- diate representations (B´erard et al., 2016; Weiss et al., 2017). The main advantages of direct ST sys- tems are: i) the access to information not present in the text (e.g. prosody, vocal characteristics of the speaker) during the translation phase, ii) a reduced latency, iii) a simpler and easier to manage architec- ture (only one model has to be maintained), which iv) avoids error propagation across components. In both paradigms (cascade and direct), the audio is commonly represented as a sequence of vectors obtained with a Mel filter bank. These vectors are collected with a high frequency, typically one every 10 ms. The resulting sequences are much longer than the corresponding textual ones (usually by a factor of ~10). The sequence length is problem- atic both for RNN (Elman, 1990) and Transformer (Vaswani et al., 2017) architectures. Indeed, RNNs fail to represent long-range dependencies (Bengio et al., 1993) and the Transformer has a quadratic memory complexity in the input sequence length, which makes training on long sequences prohibitive due to its memory footprint. For this reason, archi- tectures proposed for direct ST/ASR reduce the in- put length either with convolutional layers (B´erard et al., 2018; Di Gangi et al., 2019) or by stacking and downsampling consecutive samples (Sak et al., 2015). However, these fixed-length reductions of the …

Content truncated to 500 words