Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation

Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation Ali Can Kocabiyikoglu⋆, Laurent Besacier⋆, Olivier Kraif† ⋆LIG, UGA, G-INP, CNRS, INRIA, Grenoble, France †LIDILEM, UGA, Grenoble, France Univ. Grenoble Alpes, F-38400, Saint-Martin d’Heres alicankocabiyikoglu@gmail.com, laurent.besacier@univ-grenoble-alpes.fr, olivier.kraif@univ-grenoble-alpes.fr Abstract Recent works in spoken language translation (SLT) have attempted to build end-to-end speech-to-text translation without using source language transcription during learning or decoding. However, while large quantities of parallel texts (such as Europarl, OpenSubtitles) are available for training machine translation systems, there are no large (>100h) and open source parallel corpora that include speech in a source language aligned to text in a target language. This paper tries to fill this gap by augmenting an existing (monolingual) corpus: LibriSpeech. This corpus, used for automatic speech recognition, is derived from read audiobooks from the LibriVox project, and has been carefully segmented and aligned. After gathering French e-books corresponding to the English audio-books from LibriSpeech, we align speech segments at the sentence level with their respective translations and obtain 236h of usable parallel data. This paper presents the details of the processing as well as a manual evaluation conducted on a small subset of the corpus. This evaluation shows that the automatic alignments scores are reasonably correlated with the human judgments of the bilingual alignment quality. We believe that this corpus (which is made available online) is useful for replicable experiments in direct speech translation or more general spoken language translation experiments. Keywords: direct speech translation, bilingual alignment, librispeech corpus 1. Introduction Attention-based encoder-decoder approaches have been very successful in Machine Translation (Bahdanau et al., 2014), and have shown promising results in End- to-End Speech Translation (B´erard et al., 2016; Weiss et al., 2017) (translation from raw speech, without any intermediate transcription). End-to-End speech transla- tion is also attractive for language documentation, which often uses corpora made of audio recordings aligned with their translation in another language (no tran- script in the source language) (Blachon et al., 2016; Adda et al., 2016; Anastasopoulos and Chiang, 2017). However, while large quantities of parallel texts (such as Europarl, OpenSubtitles) are available for training (text) machine translation systems, there are no large (>100h) and open source parallel corpora that include speech in a source language aligned to text in a target language. For End-to-End speech translation, only a few parallel corpora are publicly available. For example, Fisher and Callhome Spanish-English corpora provide 38 hours of speech tran- scriptions of telephonic conversations aligned with their translations (Post et al., 2013). However, these corpora are only medium size and contain low-bandwidth record- ings. Microsoft Speech Language Translation (MSLT) cor- pus also provides speech aligned to translated text. Speech is recorded through Skype for English, German and French (Federmann and Lewis, 2016). But this corpus is again rather small (less than 8h per language). Paper contributions. Our objective is to provide a large corpus for direct speech translation evaluation which is an order of magnitude bigger than existing corpora described in the introduction. For this, we propose …

Content truncated to 500 words