The University of Helsinki Submission to the IWSLT2020 Offline SpeechTranslation Task

Proceedings of the 17th International Conference on Spoken Language Translation (IWSLT), pages 95–102 July 9-10, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 95 The University of Helsinki submission to the IWSLT2020 Offline Speech Translation Task Ra´ul V´azquez, Mikko Aulamo, Umut Sulubacak, J¨org Tiedemann University of Helsinki {name.surname}@helsinki.fi Abstract This paper describes the University of Helsinki Language Technology group’s participation in the IWSLT 2020 offline speech translation task, addressing the translation of English au- dio into German text. In line with this year’s task objective, we train both cascade and end- to-end systems for spoken language transla- tion. We opt for an end-to-end multitasking ar- chitecture with shared internal representations and a cascade approach that follows a stan- dard procedure consisting of ASR, correction, and MT stages. We also describe the exper- iments that served as a basis for the submit- ted systems. Our experiments reveal that mul- titasking training with shared internal repre- sentations is not only possible but allows for knowledge-transfer across modalities. 1 Introduction An effective solution for performing spoken lan- guage translation (SLT) must deal with the evident challenge of transferring the implicit semantics be- tween audio and text modalities. An end-to-end SLT system must hence appropriately address this problem while simultaneously performing accurate machine translation (MT) (Sulubacak et al., 2018). In last year’s IWSLT challenge, both end-to- end and cascade systems yielded similar results (Niehues et al., 2019). It follows that this year’s IWSLT offline speech translation challenge focuses on whether ”the cascaded solution is still the dom- inant technology in spoken language translation” (Ansari et al., 2020). For our participation on this task, we train both cascade and end-to-end sys- tems for SLT. For the end-to-end system, we use a multimodal approach trained in a multitask fashion, which maps the internal representations of different encoders into a shared space before decoding. For the cascade approach, we use a pipeline of three stages: (i) automatic speech recognition (ASR), (ii) punctuation and letter-case restoration, and (iii) MT. We focus on exploiting the knowledge-transfer capabilities of a multitasking architecture based on language-specific encoders-decoders (Lu et al., 2018; Schwenk and Douze, 2017; Luong et al., 2016). This idea has been proposed and studied in the multilingual scenario (V´azquez et al., 2020; Subramanian et al., 2018; Firat et al., 2017), how- ever, we adapt it to be used in a multimodal sce- nario. Regarding different modalities (in this case, audio and text) as different languages when train- ing the model, allows us to employ a cross-modal intermediate shared layer for performing SLT in an end-to-end fashion. By jointly training this layer, we aim for the the model to combine the semantic information provided in the text-to-text MT tasks with the ability to generate text from audio in the ASR tasks. 2 Proposed Systems End-to-end SLT We use an inner-attention based architecture pro- posed by V´azquez et al. (2020). In a nutshell, it follows the conventional structure of an encoder- decoder model of MT (Bahdanau et al., 2015; Lu- ong et …

Content truncated to 500 words