The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Task

Proceedings of the 18th International Conference on Spoken Language Translation, pages 92–99 Bangkok, Thailand (Online), August 5–6, 2021. ©2021 Association for Computational Linguistics 92 The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Task Chen Xu1, Xiaoqian Liu1, Xiaowen Liu1, Laohu Wang1, Canan Huang1, Tong Xiao1,2, Jingbo Zhu1,2 1NLP Lab, School of Computer Science and Engineering Northeastern University, Shenyang, China 2NiuTrans Research, Shenyang, China {xuchenneu,liuxiaoqianneu,liuxiaowenneu}@outlook.com, {tigerneu,huangcananneu}@outlook.com, {xiaotong,zhujingbo}@mail.neu.edu.cn Abstract This paper describes the submission of the Ni- uTrans end-to-end speech translation system for the IWSLT 2021 offline task, which trans- lates from the English audio to German text di- rectly without intermediate transcription. We use the Transformer-based model architecture and enhance it by Conformer, relative position encoding, and stacked acoustic and textual en- coding. To augment the training data, the En- glish transcriptions are translated to German translations. Finally, we employ ensemble de- coding to integrate the predictions from sev- eral models trained with the different datasets. Combining these techniques, we achieve 33.84 BLEU points on the MuST-C En-De test set, which shows the enormous potential of the end-to-end model. 1 Introduction Speech translation (ST) aims to learn models that can predict, given some speech in the source lan- guage, the translation into the target language. End- to-end (E2E) approaches have become popular re- cently for its ability to free designers from cascad- ing different systems and shorten the pipeline of translation (Duong et al., 2016; Berard et al., 2016; Weiss et al., 2017). This paper describes the sub- mission of the NiuTrans E2E ST system for the IWSLT 2021 (Anastasopoulos et al., 2021) offline task, which translates from the English audio to the German text translation directly without intermedi- ate transcription. Our baseline model is based on the DLCL Trans- former (Vaswani et al., 2017; Wang et al., 2019) model with Connectionist Temporal Classification (CTC) (Graves et al., 2006) loss on the encoders (Bahar et al., 2019). We enhance it with the supe- rior model architecture Conformer (Gulati et al., 2020), relative position encoding (RPE) (Shaw et al., 2018), and stacked acoustic and textual en- coding (SATE) (Xu et al., 2021). To augment the training data, the English transcriptions of the auto- matic speech recognition (ASR) and speech trans- lation corpora are translated to the German trans- lation. Finally, we employ the ensemble decoding method to integrate the predictions from multiple models (Wang et al., 2018) trained with the differ- ent datasets. This paper is structured as follows. The training data is summarized in Section 2, then we describe the model architecture in Section 3 and data aug- mentation in Section 4. We present the ensemble decoding method in Section 5. The experimental settings and final results are shown in Section 6. 2 Training Data Our system is built under the constraint condition. The training data can be divided into three cate- gories: ASR, MT, and ST corpora1. ASR corpora. ASR corpora are used to gener- ate synthetic speech translation data. We only use the Common Voice (Ardila et …

Content truncated to 500 words