Dealing with training and test segmentation mismatch: FBK@IWSLT2021

Proceedings of the 18th International Conference on Spoken Language Translation, pages 84–91 Bangkok, Thailand (Online), August 5–6, 2021. ©2021 Association for Computational Linguistics 84 Dealing with training and test segmentation mismatch: FBK@IWSLT2021 Sara Papi1,2, Marco Gaido1,2, Matteo Negri1, Marco Turchi1 1Fondazione Bruno Kessler, Trento, Italy 2University of Trento, Italy {spapi|mgaido|negri|turchi}@fbk.eu Abstract This paper describes FBK’s system submis- sion to the IWSLT 2021 Offline Speech Trans- lation task. We participated with a direct model, which is a Transformer-based architec- ture trained to translate English speech audio data into German texts. The training pipeline is characterized by knowledge distillation and a two-step fine-tuning procedure. Both knowl- edge distillation and the first fine-tuning step are carried out on manually segmented real and synthetic data, the latter being generated with an MT system trained on the available cor- pora. Differently, the second fine-tuning step is carried out on a random segmentation of the MuST-C v2 En-De dataset. Its main goal is to reduce the performance drops occurring when a speech translation model trained on manu- ally segmented data (i.e. an ideal, sentence- like segmentation) is evaluated on automati- cally segmented audio (i.e. actual, more re- alistic testing conditions). For the same pur- pose, a custom hybrid segmentation procedure that accounts for both audio content (pauses) and for the length of the produced segments is applied to the test data before passing them to the system. At inference time, we com- pared this procedure with a baseline segmenta- tion method based on Voice Activity Detection (VAD). Our results indicate the effectiveness of the proposed hybrid approach, shown by a reduction of the gap with manual segmentation from 8.3 to 1.4 BLEU points. 1 Introduction Speech translation (ST) is the task of translating a speech uttered in one language into its textual representation in a different language. Unlike si- multaneous ST, where the audio is translated as soon as it is produced, in the offline setting the au- dio is entirely available and translated at once. In continuity with the last two rounds of the IWSLT evaluation campaign (Niehues et al., 2019; Ansari et al., 2020), the IWSLT2021 Offline Speech Trans- lation task (Anastasopoulos et al., 2021) focused on the translation into German of English audio data extracted from TED talks. Participants could approach the task either with a cascade architecture or with a direct end-to-end system. The former rep- resents the traditional pipeline approach (Stentiford and Steer, 1988; Waibel et al., 1991) comprising an automatic speech recognition (ASR) followed by a machine translation (MT) component. The latter (B´erard et al., 2016; Weiss et al., 2017) relies on a single neural network trained to translate the input audio into target language text bypassing any intermediate symbolic representation steps. The two paradigms have advantages and disad- vantages. Cascade architectures have historically guaranteed higher translation quality (Niehues et al., 2018, 2019) thanks to the large corpora avail- able to train their ASR and MT sub-components. However, a well-known drawback of pipelined so- lutions is represented by error propagation: …

Content truncated to 500 words