MLSUM: The Multilingual Summarization Corpus

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8051–8067, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8051 MLSUM: The Multilingual Summarization Corpus Thomas Scialom⋆‡, Paul-Alexis Dray⋆, Sylvain Lamprier‡, Benjamin Piwowarski⋄‡, Jacopo Staiano⋆ ⋄ CNRS, France ‡ Sorbonne Université, CNRS, LIP6, F-75005 Paris, France ⋆ reciTAL, Paris, France {thomas,paul-alexis,jacopo}@recital.ai {sylvain.lamprier,benjamin.piwowarski}@lip6.fr Abstract We present MLSUM, the first large-scale Mul- tiLingual SUMmarization dataset. Obtained from online newspapers, it contains 1.5M+ ar- ticle/summary pairs in five different languages – namely, French, German, Spanish, Russian, Turkish. Together with English news articles from the popular CNN/Daily mail dataset, the collected data form a large scale multilingual dataset which can enable new research direc- tions for the text summarization community. We report cross-lingual comparative analyses based on state-of-the-art systems. These high- light existing biases which motivate the use of a multi-lingual dataset. 1 Introduction The document summarization task requires sev- eral complex language abilities: understanding a long document, discriminating what is relevant, and writing a short synthesis. Over the last few years, advances in deep learning applied to NLP have con- tributed to the rising popularity of this task among the research community (See et al., 2017; Kry´s- ci´nski et al., 2018; Scialom et al., 2019). As with other NLP tasks, the great majority of available datasets for summarization are in English, and thus most research efforts focus on the English language. The lack of multilingual data is partially countered by the application of transfer learning techniques enabled by the availability of pre-trained multilin- gual language models. This approach has recently established itself as the de-facto paradigm in NLP (Guzmán et al., 2019). Under this paradigm, for encoder/decoder tasks, a language model can first be pre-trained on a large corpus of texts in multiple languages. Then, the model is fine-tuned in one or more pivot languages for which the task-specific data are available. At inference, it can still be applied to the different lan- guages seen during the pre-training. Because of the dominance of English for large scale corpora, English naturally established itself as a pivot for other languages. The availability of multilingual pre-trained models, such as BERT multilingual (M- BERT), allows to build models for target languages different from training data. However, previous works reported a significant performance gap be- tween English and the target language, e.g. for classification (Conneau et al., 2018) and Question Answering (Lewis et al., 2019) tasks. A similar approach has been recently proposed for summa- rization (Chi et al., 2019) obtaining, again, a lower performance than for English. For specific NLP tasks, recent research efforts have produced evaluation datasets in several target languages, allowing to evaluate the progress of the field in zero-shot scenarios. Nonetheless, those ap- proaches are still bound to using training data in a pivot language for which a large amount of anno- tated data is available, usually English. This pre- vents investigating, for instance, whether a given model is as fitted for a specific language as for any other. Answers to …

Content truncated to 500 words