The MeMAD Submission to the IWSLT 2018 Speech Translation Task

The MeMAD Submission to the IWSLT 2018 Speech Translation Task Umut Sulubacak ∗ J¨org Tiedemann ∗ Aku Rouhe † Stig-Arne Gr¨onroos † Mikko Kurimo † ∗ Department of Digital Humanities / HELDIG University of Helsinki, Finland {umut.sulubacak | jorg.tiedemann}@helsinki.fi † Department of Signal Processing and Acoustics Aalto University, Finland {aku.rouhe | stig-arne.gronroos | mikko.kurimo}@aalto.fi Abstract This paper describes the MeMAD project entry to the IWSLT Speech Translation Shared Task, addressing the translation of English audio into German text. Between the pipeline and end-to-end model tracks, we participated only in the former, with three contrastive systems. We tried also the latter, but were not able to finish our end-to-end model in time. All of our systems start by transcribing the audio into text through an automatic speech recognition (ASR) model trained on the TED-LIUM English Speech Recognition Corpus (TED-LIUM). Afterwards, we feed the transcripts into English-German text-based neural machine transla- tion (NMT) models. Our systems employ three different translation models trained on separate training sets compiled from the English-German part of the TED Speech Transla- tion Corpus (TED-TRANS) and the OPENSUBTITLES2018 section of the OPUS collection. In this paper, we also describe the experiments leading up to our final systems. Our experiments indicate that us- ing OPENSUBTITLES2018 in training significantly improves translation performance. We also experimented with various pre- and postprocessing routines for the NMT module, but we did not have much success with these. Our best-scoring system attains a BLEU score of 16.45 on the test set for this year’s task. 1. Introduction The evident challenge of speech translation is the transfer of implicit semantics between two different modalities. An end-to-end solution to this task must deal with the challenge posed by intermodality simultaneously with that of inter- lingual transfer. In a traditional pipeline approach, while speech-to-text transcription is abstracted from translation, there is then the additional risk of error transfer between the two stages. The MeMAD project1 aims at multilingual 1https://www.memad.eu/ description and search in audiovisual data. For this reason, multimodal translation is of great interest to the project. Our pipeline submission to this year’s speech transla- tion task incorporates one ASR model and three contrastive NMT models. For the ASR module, we trained a time-delay neural network (TDNN) acoustic model using the Kaldi toolkit [1] on the provided TED-LIUM speech recognition corpus [2]. We used the transformer implementation of Mar- ianNMT [3] to train our NMT models. For these models, we used contrastive splits of data compiled from two differ- ent sources: The n-best decoding hypotheses of the TED- TRANS [4] in-domain speech data, and a version of the OPENSUBTITLES2018 [5] out-of-domain text data (SUBS), further “translated” to an ASR-like format (SUBS-ASR) using a sequence-to-sequence NMT model. The primary system in our submission uses the NMT model trained on the whole data including SUBS-ASR, whereas one of the two contrastive systems uses the original SUBS before the conversion to an ASR-like format, and the other omits OPENSUBTITLES2018 altogether. We provide further details about the ASR module in Sec- …

Content truncated to 500 words