MuST-Cinema: a Speech-to-Subtitles corpus

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3727–3734 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3727 MuST-Cinema: a Speech-to-Subtitles corpus Alina Karakanta1,2, Matteo Negri1, Marco Turchi1 1 Fondazione Bruno Kessler, Via Sommarive 18, Povo, Trento - Italy 2 University of Trento, Italy {akarakanta,negri,turchi}@fbk.eu Abstract Growing needs in localising audiovisual content in multiple languages through subtitles call for the development of automatic solutions for human subtitling. Neural Machine Translation (NMT) can contribute to the automatisation of subtitling, facilitating the work of human subtitlers and reducing turn-around times and related costs. NMT requires high-quality, large, task-specific training data. The existing subtitling corpora, however, are missing both alignments to the source language audio and important information about subtitle breaks. This poses a significant limitation for developing efficient automatic approaches for subtitling, since the length and form of a subtitle directly depends on the duration of the utterance. In this work, we present MuST-Cinema, a multilingual speech translation corpus built from TED subtitles. The corpus is comprised of (audio, transcription, translation) triplets. Subtitle breaks are preserved by inserting special symbols. We show that the corpus can be used to build models that efficiently segment sentences into subtitles and propose a method for annotating existing subtitling corpora with subtitle breaks, conforming to the constraint of length. Keywords: Subtitling, Neural Machine Translation, Audiovisual Translation 1. Introduction In the past few years, the audiovisual sector has witnessed an unprecedented growth, mostly due to the immense amounts of videos on-demand becoming available. In order to reach global audiences, audiovisual content providers localise their content into the language of the target audience. This has led to a “subtitling boom”, since there is a huge need for offering high-quality subtitles into dozens of languages in a short time. However, the workflows for subtitling are com- plex; translation is only one step in a long pipeline consisting of transcription, timing (also called spotting) and editing. Subtitling currently heavily relies on human work and hence manual approaches are laborious and costly. Therefore, there is ample ground for developing automatic solutions for efficiently providing subtitles in multiple languages, re- ducing human workload and the overall costs of spreading audiovisual content across cultures via subtitling. Recent developments in Neural Machine Translation (NMT) have opened up possibilities for processing inputs other than text within a single model component. This is particularly relevant for subtitling, where the translation depends not only on the source text, but also on acoustic and visual information. For example, in Multimodal NMT (Barrault et al., 2018) the input can be both text and image, and Spoken Language NMT directly receives audio as input (Niehues et al., 2019). These developments can be leveraged in order to reduce the effort involved in subtitling. Training NMT systems, however, requires large amounts of high-quality parallel data, representative of the task. A recent study (Karakanta et al., 2019) questioned the confor- mity of existing subtitling corpora to subtitling constraints. The authors suggested that subtitling …

Content truncated to 500 words