Breaking the Data Barrier: Towards Robust Speech Translation via Adversarial Stability Training

Breaking the Data Barrier: Towards Robust Speech Translation via Adversarial Stability Training Qiao Cheng, Meiyuan Fang, Yaqian Han, Jin Huang, Yitao Duan NetEase Youdao Information Technology (Beijing) Co., LTD. Beijing, China {chengqiao, fangmeiyuan, hanyaqian, huangjin, duan}@rd.netease.com Abstract In a pipeline speech translation system, automatic speech recognition (ASR) system will transmit errors in recognition to the downstream machine translation (MT) system. A stan- dard machine translation system is usually trained on paral- lel corpus composed of clean text and will perform poorly on text with recognition noise, a gap well known in speech translation community. In this paper, we propose a training architecture which aims at making a neural machine trans- lation model more robust against speech recognition errors. Our approach addresses the encoder and the decoder simul- taneously using adversarial learning and data augmentation, respectively. Experimental results on IWSLT2018 speech translation task show that our approach can bridge the gap between the ASR output and the MT input, outperforms the baseline by up to 2.83 BLEU on noisy ASR output, while maintaining close performance on clean text. 1. Introduction A pipeline spoken language translation (SLT) system inte- grates two main modules: source language speech recogni- tion (ASR) and source-to-target text translation (MT). The ASR system transforms the input audios into automatic tran- scriptions which the MT system translates into texts in the target language. However, since the two systems are trained separately using different parallel corpora, there is a mis- match between the output of ASR and the expected input of MT system. The former does not provide punctuation or case information, and contains recognition errors such as omission, repetition, homophones errors, etc. In contrast, a standard MT system is usually trained on written text which is much cleaner. Although an end-to-end speech translation model [1, 2] seems to be a promising solution to alleviate the problem of error propagation, due to the scarcity of training data in the form of speech aligned with text and translation, currently the pipeline system is still the best performing model so far. Prior works attempting to enhance the robustness of speech translation suffer from two main limitations: they either require training data in the form of speech aligned with either translation or both transcription and translation, or use artificially generated noise. The first problem is faced by end-to-end models [1, 2] and [3, 4]. Such speech-to- transcription-to-translation data is extremely costly to ac- quire since it requires two human (even expert) labeling pro- cesses for each sentence. We are only aware of four such cor- pora that are publicly available: Fisher and Callhome Span- ish–English Speech Translation Corpus [5], Speech transla- tion TED corpus [6], MaSS [7] and MuST-C [8]. All of them are far less than what is necessary for training even a single ASR or MT model. The works relying on artificial noise in- clude [9,10] etc. They only achieve minor improvements on noisy input but harm the translation quality on clean text. On the other hand, we observe that, when …

Content truncated to 500 words