End-to-End Offline Speech Translation System for IWSLT 2020 using Modality Agnostic Meta-Learning

Proceedings of the 17th International Conference on Spoken Language Translation (IWSLT), pages 73–79 July 9-10, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 73 End-to-End Offline Speech Translation System for IWSLT 2020 using Modality Agnostic Meta-Learning Nikhil Kumar Lakumarapu*, Beomseok Lee*, Sathish Indurthi, Houjeung Han, Mohd Abbas Zaidi, Sangha Kim Next AI Solution Lab, Samsung Research, Seoul, South Korea {n07.kumar, bsgunn.lee, s.indurthi, h.j.han, abbas.zaidi, sangha01.kim}@samsung.com Abstract In this paper, we describe the system sub- mitted to the IWSLT 2020 Offline Speech Translation Task. We adopt the Transformer architecture coupled with the meta-learning approach to build our end-to-end Speech- to-Text Translation (ST) system. Our meta-learning approach tackles the data scarcity of the ST task by leveraging the data available from Automatic Speech Recognition (ASR) and Machine Translation (MT) tasks. The meta-learning approach combined with synthetic data augmentation techniques improves the model performance significantly and achieves BLEU scores of 24.58, 27.51, and 27.61 on IWSLT test 2015, MuST-C test, and Europarl-ST test sets respectively. 1 Introduction The goal of the IWSLT 2020 Offline Speech Trans- lation challenge(Ansari et al., 2020) is to check the feasibility of end-to-end models for translat- ing audio speech of one language into text of a different target language. The success of end-to- end neural models for ASR (Graves et al., 2013) and MT (Bahdanau et al., 2015) inspired to build end-to-end neural models for the more challeng- ing Speech-to-Text translation (ST) task (B´erard et al., 2016). Traditionally the ST systems are built by cascading ASR and MT systems (Ney, 1999). However, the cascaded system suffers from error propagation, latency, and memory requirement is- sues. Although these issues can be addressed using end-to-end ST models, it is hard to collect such data for training these models. In this work, we build an end-to-end ST sys- tem which not only addresses the issues of a cas- caded system but also works with limited training data. The proposed system is fine-tuned towards * The two authors contributed equally to this paper IWSLT 2020 Offline Speech-Translation Task 1. However, the proposed training strategies and the data augmentation techniques can be adopted into existing and future ST models. We adopt the meta- learning approach proposed for ST task (Indurthi et al., 2019) to train our system. The meta-learning based training approach not only allows us to lever- age huge amounts of training data available in ASR and MT tasks but also helps to find a good initial- ization point for the target ST task. We conduct several experiments involving ASR, MT, and ST corpora to test our model performance on the IWSLT 2020, MuST-C, and Europarl-ST English-German (En-De) ST tasks. Our experi- ments reveal that the proposed model trained us- ing the meta-learning approach achieves significant performance gains over the model which only uti- lizes the ST data for training. Our model achieves 4.81, 5.37, and 8.46 BLEU score improvements on IWSLT test 2015, MuST-C test, Europarl-ST test sets compared to the models trained without using the meta-learning approach for training. Our best …

Content truncated to 500 words