The USTC-NEL Speech Translation system at IWSLT 2018

The USTC-NEL Speech Translation system at IWSLT 2018 Dan Liu1,2, Junhua Liu1,2, Wu Guo1 Shifu Xiong2, Zhiqiang Ma2, Rui Song2, Chongliang Wu2, Quan Liu2 University of Science and Technology of China 1 IFLYTEK Co. LTD. 2 {danliu,jhliu}@mail.ustc.edu.cn wuguo@ustc.edu.cn {danliu, jhliu, sfxiong, zqma2, ruisong, clwu4, quanliu}@iflytek.com Abstract This paper describes the USTC-NEL (short for ”National En- gineering Laboratory for Speech and Language Information Processing University of science and technology of china”) system to the speech translation task of the IWSLT Eval- uation 2018. The system is a conventional pipeline sys- tem which contains 3 modules: speech recognition, post- processing and machine translation. We train a group of hybrid-HMM models for our speech recognition, and for ma- chine translation we train transformer based neural machine translation models with speech recognition output style text as input. Experiments conducted on the IWSLT 2018 task in- dicate that, compared to baseline system from KIT, our sys- tem achieved 14.9 BLEU improvement. 1. Introduction Conventional speech translation systems consist of three components: source-language automatic speech recognition (ASR), post-processing over ASR outputs, and source-to- target text translation. This pipeline system suffers from er- ror accumulation, which means speech recognition and trans- lation models trained separately may perform well individu- ally, but do not work well together because their error surface do not compose well [1]. In the most recent years, end-to-end speech translation based on encoder-decoder with attention mechanisms has been very promising for reducing accumulated errors [2, 1, 3]. However, parallel speech data is much smaller than those available to train text-based machine translation (MT) sys- tems, particularly neural systems that needs to learn a rela- tively large parameters. As a result, an end-to-end speech translation system can often outperform pipeline systems with same training data, but is hard to beat pipeline system with dozens of training data [1]. In addition, to translate very long speech (e.g. translate a full talk), an end-to-end system must rely on voice activity detection (VAD) method to split raw audio into sentence-like fragments, in which mis-segmented sentence fragments are very likely to cause serious translation errors. Therefore, for pipeline systems, sentence re-segmentation based on ASR results may be done in post-processing step, which can im- prove performance significantly [4]. To reduce the error accumulation of pipeline systems, we introduce a data augmentation based solution to train trans- lation model with ASR results as source directly, instead of normalize ASR results (e.g. insert punctuations, normaliza- tion for case, numerals, etc.) in post-processing. Text nor- malization cannot bring any new information, it just pro- duces texts that translation system likes, and this may lead to additional errors. In our experiments, the data augmenta- tion based solution performs significantly better than pipeline system with text normalization and end-to-end speech trans- lation system. This paper is organized as follows. We first describe the processing for speech and text training data in Section 2, fol- lowing is our full system and training details. Our experi- ments are presented in Section 4. 2. Data Processing We …

Content truncated to 500 words