The JHU/KyotoU Speech Translation System for IWSLT 2018

The JHU/KyotoU Speech Translation System for IWSLT 2018 Hirofumi Inaguma†, Xuan Zhang, Zhiqi Wang, Adithya Renduchintala, Shinji Watanabe, Kevin Duh Center for Language and Speech Processing, Johns Hopkins University, Baltimore, USA †Graduate School of Informatics, Kyoto University, Japan {xuanzhang,zwang132,adithya.renduchintala,shinjiw}@jhu.edu inaguma@sap.ist.i.kyoto-u.ac.jp, kevinduh@cs.jhu.edu Abstract This paper describes the Johns Hopkins University (JHU) and Kyoto University submissions to the Speech Translation evaluation campaign at IWSLT2018. Our end-to-end speech translation systems are based on ESPnet and implements an attention-based encoder-decoder model. As comparison, we also experiment with a pipeline system that uses independent neural network systems for both the speech transcription and text translation components. We find that a transfer learn- ing approach that bootstraps the end-to-end speech transla- tion system with speech transcription system’s parameters is important for training on small datasets. 1. Introduction We report on our efforts on the IWSLT 2018 Speech Trans- lation task. The goal of the 2018 task is to build and evalu- ate English-to-German speech translation systems on the do- main of lectures and TED talks. We build two systems: • Pipeline System: English (EN) speech transcription system using a joint CTC-attention model (Section 3.1), followed by a English-to-German (EN-DE) text translation system using a RNN-based sequence-to- sequence model (Section 3.3). • End-to-End System: English-to-German (EN-DE) speech translation system using an RNN-based sequence-to-sequence model transferred from the joint CTC-attention model (Section 4). The main challenge is to develop end-to-end neural sys- tems that are trainable given the small amount of data (of English speech matched to German text). We find that boot- strapping the end-to-end system with the parameters of an English-only speech transcription system (i.e. ASR of En- glish speech to English text) was helpful. Generally, we are interested in comparing the relative merits of end-to-end vs. pipeline approaches. Currently, our pipeline system outperforms the end-to-end system, even when trained on the same number of utterances, suggesting that there is much room for future work in end-to-end mod- els. † Work carried out as a visiting scholar at JHU. 2. Data We build our systems on the following provided corpora: 1. Speech-Translation TED corpus (ST TED): This data contains English speech (EN-s), the corresponding English transcription (EN-t), as well as the German translation (DE-t). We use this to train both pipeline and end-to-end systems. In particular, (EN-s,EN-t) is used to train the pipeline’s speech transcription com- ponent (Section 3.1); (EN-t, DE-t) is used to train the pipeline’s text translation component (Section 3.3); and (EN-s, DE-t) is used to train the end-to-end sytem (Section 4). 2. TED LIUM corpus (TEDLIUM2): This data contains English speech (EN-s) and their English transcrip- tion (EN-t). We use this as additional data to train the pipeline system’s speech transcription component, which is also used to initialize the end-to-end system. 3. WMT 2018 data, filtered to the TED domain us- ing Moore-Lewis data selection [1] (WMT-Filtered): We trained 5-gram Engish language models on TED (LMT ED) and a random sample of the WMT data (LMW MT ), then selected the top 1 …

Content truncated to 500 words