End-to-end Speech Translation System Description of LIT for IWSLT 2019

End-to-end Speech Translation System Description of LIT for IWSLT 2019 Mei Tu, Wei Liu, Lijie Wang, Xiao Chen, Xue Wen Speech Lab & Language Understanding Lab of Language Intelligence Team, Beijing {mei.tu,wei.liu,lijie.wang,xiao.chen,xue.wen}@samsung.com Abstract This paper describes our end-to-end speech translation system for the speech translation task of lectures and TED talks from English to German for IWSLT Evaluation 2019. We propose layer-tied self-attention for end-to-end speech translation. Our method takes advantage of sharing weights of speech encoder and text decoder. The representation of source speech and the representation of target text are coordinated layer by layer, so that the speech and text can learn a better alignment during the training procedure. We also adopt data augmentation to enhance the parallel speech-text corpus. The En-De experimental results show that our best model achieves 17.68 on tst2015. Our ASR achieves WER of 6.6% on TED-LIUM test set. The En-Pt model can achieve about 11.83 on the MuST-C dev set. 1. Introduction End-to-end Speech Translation is a promising task that attracts a lot of attention in recent years [1-5,11]. Compared to a traditional cascaded system [9,10] that performs ASR and MT separately, the end-to-end speech translation model complies with the encoder-decoder structure, which has advantages including 1) avoiding compounding errors between ASR and MT; 2) performing lower latency without the separate ASR component. However, the translation quality speech translation is still hard to achieve the SOTA text translation performance. The main reason lies that the parallel speech-text translation corpus is hard to obtain, so that the learning process becomes a low- resource learning task. Previous studies [1-4] tend to address the training data sparseness by introducing multi-task learning. With multi-task learning framework, the translation model can get benefits from ASR corpus and MT corpus separately. The study [5] directly generates the speech data given text data by TTS system, but the generated data may face the low diversity in speaker voices in the synthetic data. The state-of-the-art speech translation models mostly adopt the architecture in [1]. The encoder consists of CNN layers for down-sampling and stacked RNN cells to deeply represent the speech features, while the decoder performs attentive RNN under autoregressive mechanism. Roughly speaking, the encoder is for projecting the down-sampled source speech feature into a deep semantic space. The decoder is for generating text tokens based on the speech deep representation. When the encoder and decoder networks become deep, the information from source low-level layers is hard to pass to the high level in the decoder, even with an attention mechanism, which causes alignment information reduction. The CNN layers aggravate the information reduction after down- sampling. The information reduction problem was discussed in the previous works [7,8] in text translation. They coordinate the encoder layers and decoder layers, so that the decoder neuron can also refer to the low-level neuron of source sentences to generate target sentences. Inspired by the mentioned works, in this paper, we introduce a layer-wise tied structure for end-to- end speech translation. We build connections between speech …

Content truncated to 500 words