CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2541–2564 July 9-14, 2023 ©2023 Association for Computational Linguistics CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs Abhik Bhattacharjee1∗, Tahmid Hasan1∗, Wasi Uddin Ahmad2, Yuan-Fang Li3, Yong-Bin Kang4, Rifat Shahriyar1 Bangladesh University of Engineering and Technology (BUET)1, University of California, Los Angeles2, Monash University3, Swinburne University of Technology4 {tahmidhasan,rifat}@cse.buet.ac.bd, abhik@ra.cse.buet.ac.bd Abstract We present CrossSum, a large-scale cross- lingual summarization dataset comprising 1.68 million article-summary samples in 1,500+ lan- guage pairs. We create CrossSum by aligning parallel articles written in different languages via cross-lingual retrieval from a multilingual abstractive summarization dataset and perform a controlled human evaluation to validate its quality. We propose a multistage data sampling algorithm to effectively train a cross-lingual summarization model capable of summariz- ing an article in any target language. We also introduce LaSE, an embedding-based metric for automatically evaluating model-generated summaries. LaSE is strongly correlated with ROUGE and, unlike ROUGE, can be reliably measured even in the absence of references in the target language. Performance on ROUGE and LaSE indicate that our proposed model consistently outperforms baseline models. To the best of our knowledge, CrossSum is the largest cross-lingual summarization dataset and the first ever that is not centered around En- glish. We are releasing the dataset, training and evaluation scripts, and models to spur future research on cross-lingual summariza- tion. The resources can be found at https: //github.com/csebuetnlp/CrossSum. 1 Introduction Cross-lingual summarization (hereinafter XLS) is the task of generating a summary in a target lan- guage given a source text in another language. The task is challenging as it combines summarization and translation in one task, both challenging tasks in their own right. Earlier approaches to XLS thus employed pipeline methods such as translate-then- summarize (Leuski et al., 2003) and summarize- then-translate (Wan et al., 2010). Not only are they computationally expensive, having to use multiple ∗These authors contributed equally to this work. Input Article: [...] 新型コロナウイルスに対し、様々な既存の 治療法の効果を試す世界的規模の臨床試験の一貫として、デキ サメタゾンが試された。(Dexamethasone was tested as part of a global clinical trial to test the effectiveness of various exist- ing therapies against the new coronavirus.) [...] その結果、人 工呼吸器を必要とする重症患者の致死率が3割下がり。(As a result, the case fatality rate of critically ill patients who require a ventilator is reduced by 30%.) [...] ボリス･ジョンソン 英首相は「イギリス科学界の素晴らしい成果」を歓迎し。(British Prime Minister Boris Johnson welcomed "the great achieve- ments of the British scientific community".) [...]「しかもこれ は、世界中で手に入る薬だ」("And this is a medicine available all over the world".) [...] きわめて安いステロイド剤だった (but a very cheap steroid that has been used for a long time.) Summary: িবজ্ঞানীরা বলেছন েড�ােমথােসান নােম স�া ও সহজলভয্ একিট ওষুধ কেরানাভাইরােস গুরুতর অসু� েরাগীেদর জীবন রক্ষা করেত সাহাযয্ করেব। (Scientists say a cheap and readily available drug called dexamethasone will help save the lives of critically ill patients with coronavirus.) Figure 1: A sample article-summary pair from Cross- Sum, the article is written in Japanese, and the summary is in Bengali. We translate the texts to English inside parentheses for better understanding. Words and phrases of the article relevant to the …

Content truncated to 500 words