MLLP-VRAIN UPV systems for the IWSLT 2022 Simultaneous Speech Translation and Speech-to-Speech Translation tasks

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 255 - 264 May 26-27, 2022 c⃝2022 Association for Computational Linguistics MLLP-VRAIN UPV systems for the IWSLT 2022 Simultaneous Speech Translation and Speech-to-Speech Translation tasks Javier Iranzo-Sánchez and Javier Jorge and Alejandro Pérez-González-de-Martos Adrià Giménez and Gonçal V. Garcés Díaz-Munío and Pau Baquero-Arnal Joan Albert Silvestre-Cerdà and Jorge Civera and Albert Sanchis and Alfons Juan Machine Learning and Language Processing Group Valencian Research Institute for Artificial Intelligence Universitat Politècnica de València Camí de Vera s/n, 46022 València, Spain Abstract This work describes the participation of the MLLP-VRAIN research group in the two shared tasks of the IWSLT 2022 confer- ence: Simultaneous Speech Translation and Speech-to-Speech Translation. We present our streaming-ready ASR, MT and TTS systems for Speech Translation and Synthesis from En- glish into German. Our submission combines these systems by means of a cascade approach paying special attention to data preparation and decoding for streaming inference. 1 Introduction In this paper we describe the participation of the MLLP-VRAIN research group in the shared tasks of the 19th International Conference on Spoken Language Translation (IWSLT). We participated in two shared tasks: the Simultaneous Speech Trans- lation and the (offline) Speech-to-Speech Transla- tion tasks. The translation pair for both tasks was English to German. Our submission follows the cascade approach, with individual ASR, MT and TTS components. We use common ASR and MT models for both tasks, with additional latency re- strictions for the Simultaneous task. In short, for the Simultaneous S2T task our system comprises a one-pass decoder ASR system based on the HMM- DNN approach with a chunk-based BLSTM AM combined with a Transformer LM, followed by a multi-k Transformer-based MT system. Regard- ing the S2S translation task, the aforementioned systems are followed by a non-autoregressive Conformer-based text-to-spectogram module, end- ing with a multi-band UnivNet neural vocoder to convert from the spectogram to the final audio wave. This paper is structured as follows. Section 2 describes our participation in the Simultaneous Speech Translation (ST) task: the architecture and design decisions of the ASR and MT components in our cascade system, and the evaluation of the individual components as well as the speech trans- lation system as a whole. Section 3 describes our participation in the Speech-to-Speech (S2S) Trans- lation task, paying special attention to the speaker- adaptive TTS system specifically developed for this task. Our conclusions for the shared task are drawn in Section 4. 2 Simultaneous Speech Translation 2.1 ASR System Description The acoustic model (AM) was trained using 3649 hours from resources listed in Table 4 in Ap- pendix A. The evaluation sets were those provided with MuST-C v2.0: tst-HE, tst-COMMON and dev, for the English-German language pair. To train the AM we follow our training recipe for the DNN-HMM model, thoroughly described in Jorge et al. (2022). After this training pipeline we end up with a BLSTM network with 8 bidirectional hidden layers and 512 LSTM cells per layer and direc- tion, with …

Content truncated to 500 words