The HW-TSC’s Speech to Speech Translation System for IWSLT 2022 Evaluation

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 293 - 297 May 26-27, 2022 c⃝2022 Association for Computational Linguistics The HW-TSC’s Speech to Speech Translation System for IWSLT 2022 Jiaxin Guo1, Yinglu Li1, Minghan Wang1, Xiaosong Qiao1, Yuxia Wang2, Hengchao Shang1, Chang Su1, Yimeng Chen1, Min Zhang1, Shimin Tao1, Hao Yang1, Ying Qin1 1Huawei Translation Services Center, Beijing, China 2The University of Melbourne, Melbourne, Australia {guojiaxin1,liyinglu,wangminghan,qiaoxiaosong, shanghengchao, suchang8,chenyimeng,zhangmin186, taoshimin,yanghao30,qinying}@huawei.com yuxiaw@student.unimelb.edu.au Abstract The paper presents the HW-TSC’s pipeline and results of Offline Speech to Speech Transla- tion for IWSLT 2022. We design a cascade system consisted of an ASR model, machine translation model and TTS model to convert the speech from one language into another language(en-de). For the ASR part, we find that better performance can be obtained by ensem- bling multiple heterogeneous ASR models and performing reranking on beam candidates. And we find that the combination of context-aware reranking strategy and MT model fine-tuned on the in-domain dataset is helpful to improve the performance. Because it can mitigate the problem that the inconsistency in transcripts caused by the lack of context. Finally, we use VITS model provided officially to reproduce audio files from the translation hypothesis. 1 Introduction In this year, there is only one track in the speech to speech translation task which is the English to German translation (En-De) (Anastasopoulos et al., 2022). The audio files in English are given in the dataset, and we are required to produce the audio files in German. In recent research of speech to speech task, there are basically two paradigms with respect to the system architecture, which are cas- cade and end-to-end. And the cascade pipeline composed by an ASR model, a MT model and a TTS model is commonly used, because this system is more mature than end-to-end one. The advan- tage of this pipeline is that each module of the system can be a state-of-the-art one trained on suf- ficient independent corpora. It also allows us to perform experiments with different combinations of ASR models, MT models and TTS models. But compared to end-to-end system, this cascade sys- tem may not capture all information like accent of speakers, emotion, etc. End-to-End system like S2UT is introduced in (Lee et al., 2021), which can be directly trained on Dataset Number of Utterance Duration(hrs) LibriSpeech 281,241 960.85 MuST-C 340,421 590.67 IWSLT 170,229 254.41 CoVoST 1362,422 1802.52 TEDLIUM3 268,214 453.42 Table 1: Data statistics of our ASR corpora speech to speech dataset with the help of text gen- eration as the auxiliary task. However, we didn’t adopt this approach due to the insufficiency of avail- able corpora. For the ASR model, we tried Conformer (Gulati et al., 2020), S2T Transformer (Synnaeve et al., 2019) and U2(Zhang et al., 2020), and obtained three types of ASR results. In translation, inconsistency of translation of same words in the context is a common difficulty. This is caused by the flaw of conventional trans- lation that treats each sentence independently in …

Content truncated to 500 words