The HW-TSC Video Speech Translation System at IWSLT 2020

Proceedings of the 17th International Conference on Spoken Language Translation (IWSLT), pages 187–190 July 9-10, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 187 The HW-TSC Video Speech Translation system at IWSLT 2020 Minghan Wang, Hao Yang, Yao Deng, Ying Qin, Lizhi Lei, Daimeng Wei, Hengchao Shang, Jiaxin Guo, Ning Xie, Xiaochun Li Huawei Translation Service Center, Beijing, China {wangminghan,yanghao30,dengyao3,qinying,leilizhi weidaimeng,shanghengchao,guojiaxin1, nicolas.xie,carol.lixiaochun}@huawei.com Abstract In this paper, we present details of our system in the IWSLT Video Speech Translation eval- uation. The system works in a cascade form, which contains three modules: 1) A propri- etary ASR system. 2) A disfluency correction system aims to remove interregnums or other disfluent expressions with a fine-tuned BERT and a series of rule based algorithms. 3) An NMT System based on the Transformer and trained with massive publicly available corpus is used for translation. 1 Introduction There has been great advances in Neural Machine Translations (NMT) in recent years which also promotes Multimodal translation including image (Specia et al., 2016), speech and video translation (Wang et al., 2020; Imankulova et al., 2020; Wu et al., 2019). For speech and video translation, there are basically two types of systems (i.e. cas- cade and end-to-end). Cascade systems are often composed of several independent modules, where each one can be trained with intermediate inputs and labels. End-to-end systems can be fully dif- ferentiable and trained with original multimodality data. In the IWSLT 2020 Video Speech Translation task (Ansari et al., 2020), participants are required to develop systems to translate speeches in the video from source language into target language. However, we consider that visual contexts are not necessarily important for a translation task, there- fore we only use audio as contexts, and treat it as a offline speech translation task, in addition, our system is built mainly for Chinese to English trans- lation. We choose to build our system in a cascade man- ner because training an end-to-end system requires massive aligned audio and text data, which is hard to find. On the other hand, a cascade system al- lows us to train each part separately, which is more feasible. Our system is composed with three modules. 1) A proprietary ASR system. 2) A disfluency correction system (Wang et al., 2019). 3) An NMT model based on the Transformer (Vaswani et al., 2017). 2 System Architecture 2.1 ASR For the task, we simply extract the sound tracks from videos, then feed them to our proprietary ASR system and proceed transcripts to downstream mod- ules. 2.2 Disfluency Correction A major flaw of the cascade system is the error propagation from the ASR to the NMT system. For example, interregnums like “uh”, “you know” should not be translated, or repeated words like “I wanna wanna ...” due to disfluency (Shriberg, 1994; Wang et al., 2018). To deal with this problem, we developed the disfluency correction system to de-noise the ASR outputs so that the NMT model could generate more fluent translations. The dis- fluency correction system is based on fine-tuning …

Content truncated to 500 words