Towards Unifying Multi-Lingual and Cross-Lingual Summarization

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 15127–15143 July 9-14, 2023 ©2023 Association for Computational Linguistics Towards Unifying Multi-Lingual and Cross-Lingual Summarization Jiaan Wang1∗, Fandong Meng2, Duo Zheng3, Yunlong Liang2 Zhixu Li4†, Jianfeng Qu1†and Jie Zhou2 1School of Computer Science and Technology, Soochow University, Suzhou, China 2Pattern Recognition Center, WeChat AI, Tencent Inc, China 3Beijing University of Posts and Telecommunications 4Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, China jawang.nlp@gmail.com, fandongmeng@tencent.com zhixuli@fudan.edu.cn, jfqu@suda.edu.cn Abstract To adapt text summarization to the multilingual world, previous work proposes multi-lingual summarization (MLS) and cross-lingual sum- marization (CLS). However, these two tasks have been studied separately due to the differ- ent definitions, which limits the compatible and systematic research on both of them. In this pa- per, we aim to unify MLS and CLS into a more general setting, i.e., many-to-many summariza- tion (M2MS), where a single model could pro- cess documents in any language and generate their summaries also in any language. As the first step towards M2MS, we conduct prelim- inary studies to show that M2MS can better transfer task knowledge across different lan- guages than MLS and CLS. Furthermore, we propose PISCES, a pre-trained M2MS model that learns language modeling, cross-lingual ability and summarization ability via three- stage pre-training. Experimental results indi- cate that our PISCES significantly outperforms the state-of-the-art baselines, especially in the zero-shot directions, where there is no training data from the source-language documents to the target-language summaries.1 1 Introduction The world we live in is multi-lingual. With global- ization, text resources in various languages flood the Internet, where global users can easily access their desired information. Under this background, the text summarization community presents multi- lingual summarization (MLS) and cross-lingual summarization (CLS), respectively. As shown in Figure 1, MLS aims at building a unified model to process documents in multiple languages and gen- erate summaries in the corresponding language (Gi- annakopoulos et al., 2015; Cao et al., 2020b; Hasan et al., 2021b; Wang et al., 2021; Varab and Schluter, ∗Work was done when Jiaan Wang was interning at Pat- tern Recognition Center, WeChat AI, Tencent Inc, China. †Corresponding authors. 1https://hf.co/Krystalan/PISCES MLS Model CLS Model ��� ������ ��� ������ ��� ������ ��� ������ ��� ���ℎ ��� ���ℎ ��� ������ ��� ���ℎ (a) (b) M2MS Model ��� ������ ��� ������ ��� ���ℎ (c) ��� ���ℎ ��� ������ ��� ������ Figure 1: Illustration of (a) multi-lingual summarization, (b) cross-lingual summarization and (c) many-to-many summarization. Xi and Y i denote the input document and output summary in language i, respectively. En: English; De: German; Zh: Chinese. 2021), while CLS generates a summary in the tar- get language from the given document in a different source language (Leuski et al., 2003a; Wan et al., 2010; Wan, 2011; Yao et al., 2015; Zhu et al., 2019; Ladhak et al., 2020; Perez-Beltrachini and Lap- ata, 2021; Wang et al., 2022b,d,c, 2023). Despite the close relationship between MLS and CLS (e.g., both …

Content truncated to 500 words