Multilingual Speech Translation with Unified Transformer: Huawei Noah’s Ark Lab at IWSLT 2021

Proceedings of the 18th International Conference on Spoken Language Translation, pages 149–153 Bangkok, Thailand (Online), August 5–6, 2021. ©2021 Association for Computational Linguistics 149 Multilingual Speech Translation with Unified Transformer: Huawei Noah’s Ark Lab at IWSLT 2021 Xingshan Zeng, Liangyou Li, Qun Liu Huawei Noah’s Ark Lab {zeng.xingshan,liliangyou,qun.liu}@huawei.com Abstract This paper describes the system submitted to the IWSLT 2021 Multilingual Speech Transla- tion (MultiST) task from Huawei Noah’s Ark Lab. We use a unified transformer architec- ture for our MultiST model, so that the data from different modalities (i.e., speech and text) and different tasks (i.e., Speech Recognition, Machine Translation, and Speech Translation) can be exploited to enhance the model’s abil- ity. Specifically, speech and text inputs are firstly fed to different feature extractors to ex- tract acoustic and textual features, respectively. Then, these features are processed by a shared encoder–decoder architecture. We apply sev- eral training techniques to improve the perfor- mance, including multi-task learning, task-level curriculum learning, data augmentation, etc. Our final system achieves significantly better results than bilingual baselines on supervised language pairs and yields reasonable results on zero-shot language pairs. 1 Introduction Multilingual Speech Translation (MultiST) aims to develop a single system that can directly trans- late speech in different languages into text in many other languages. Due to data scarcity of Speech Translation (ST), multilingual and multi- modal models are promising as they enable knowl- edge transferred from other languages and related tasks like Automatic Speech Recognition (ASR) or Neural Machine Translation (NMT). They also allow zero-shot translation in the settings of no di- rect parallel data. The IWSLT 2021 MultiST task is held for evaluating the performance under the circumstances. This paper describes our system for the task. We build a unified model for both speech- and text-related tasks, so that the knowledge from differ- ent modalities (speech and text) and different tasks (in this work, the tasks include ST, ASR, and NMT) can be learned together to enhance ST. Specifi- cally, our model consists of three parts – feature extractor, semantic encoder and decoder. For all the tasks, the semantic encoder and the decoder will be shared to learn unified representations. It follows the Transformer (Vaswani et al., 2017) encoder- decoder framework to learn modality-independent features and output text representations. We use the Conv-Transformer (Huang et al., 2020) as feature extractor for speech input, and the word embed- ding for text input. The extracted features are then fed to the semantic encoder regardless of the input modality. However, it is difficult for such a unified model to directly digest knowledge from diverse tasks. Therefore, we apply task-level curriculum learning for our model. We presume the ST task is more difficult than the other two tasks (ASR and NMT), as it not only requires acoustic modeling to extract speech representations, but also needs alignment knowledge between different languages for transla- tion. To this end, our training is divided into three steps – ASR and NMT pre-training, joint multi- task learning, and ST fine-tuning. What’s …

Content truncated to 500 words