ON-TRAC Consortium End-to-End Speech Translation Systems for the IWSLT 2019 Shared Task

ON-TRAC Consortium End-to-End Speech Translation Systems for the IWSLT 2019 Shared Task Ha Nguyen1, Natalia Tomashenko2, Marcely Zanon Boito1, Antoine Caubri`ere3, Fethi Bougares3, Mickael Rouvier2, Laurent Besacier1, Yannick Est`eve2 1LIG - Universit´e Grenoble Alpes, France 2LIA - Avignon Universit´e, France 3LIUM - Le Mans Universit´e, France Abstract This paper describes the ON-TRAC Consortium translation systems developed for the end-to-end model task of IWSLT Evaluation 2019 for the English→ Portuguese language pair. ON-TRAC Consortium is composed of researchers from three French academic laboratories: LIA (Avignon Univer- sit´e), LIG (Universit´e Grenoble Alpes), and LIUM (Le Mans Universit´e). A single end-to-end model built as a neural encoder-decoder architecture with attention mechanism was used for two primary submissions corresponding to the two EN-PT evaluations sets: (1) TED (MuST-C) and (2) How2. In this paper, we notably investigate impact of pooling het- erogeneous corpora for training, impact of target tokeniza- tion (characters or BPEs), impact of speech input segmenta- tion and we also compare our best end-to-end model (BLEU of 26.91 on MuST-C and 43.82 on How2 validation sets) to a pipeline (ASR+MT) approach. 1. Introduction Previous automatic speech-to-text translation (AST) sys- tems operate in two steps: source language speech recogni- tion (ASR) and source-to-target text translation (MT). How- ever, recent works have attempted to build end-to-end AST without using source language transcription during learning or decoding [1, 2] or using it at training time only [3]. Very recently several extensions of these pioneering works were introduced: low-resource AST [4], unsupervised AST [5], end-to-end speech-to-speech translation (Translatotron) [6]. Improvements of end-to-end AST were also proposed using weakly supervised data [7], or by adding a second attention mechanism [8]. This paper describes the ON-TRAC consortium auto- matic speech translation (AST) systems for the IWSLT 2019 Shared Task. ON-TRAC Consortium is composed of researchers from three French academic laboratories: LIA (Avignon Universit´e), LIG (Universit´e Grenoble Alpes), and LIUM (Le Mans Universit´e). We participated to the end-to-end model English-to- Portuguese AST task on How2 [9] and MuST-C [10] datasets. We notably try to answer to the following questions: • Question 1: does pooling heterogenous corpora (How2 and MuST-C) help the AST training? • Question 2: what is the better tokenization unit on the target side (BPE or characters)? • Question 3: considering that segmentation is an im- portant challenge of AST, what is the optimal way to segment the speech input? • Question 4: does fine-tuning increase the system’s per- formance? • Question 5: is our end-to-end AST model better than an ASR+MT pipeline? This paper is organized as follows: after briefly present- ing the data in Section 2 and after detailing our investigation on automatic speech segmentation in Section 3, we present the end-to-end speech translation systems submitted by our ON-TRAC consortium in Section 4. Section 5 summarizes what we learned from this evaluation and Section 6 con- cludes this work. 2. Data The corpora used in this work are the How2 [9] and MuST- C [10] corpora. Since we focus on English-to-Portuguese AST tasks, only the …

Content truncated to 500 words