Revisiting Cross-Lingual Summarization: A Corpus-based Study and A New Benchmark with Improved Annotation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9332–9351 July 9-14, 2023 ©2023 Association for Computational Linguistics Revisiting Cross-Lingual Summarization: A Corpus-based Study and A New Benchmark with Improved Annotation Yulong Chen1,2 Huajian Zhang2 Yijie Zhou1 Xuefeng Bai2 Yueguan Wang2 Ming Zhong3 Jianhao Yan2 Yafu Li2 Judy Li4 Michael Zhu4 Yue Zhang2,5 ∗ 1 Zhejiang University 2 Westlake University 3 UIUC 4 Sichuan Lan-bridge Information Technology Co., Ltd. 5 Westlake Institute for Advanced Study yulongchen1010@gmail.com yue.zhang@wias.org.cn Abstract Most existing cross-lingual summarization (CLS) work constructs CLS corpora by sim- ply and directly translating pre-annotated sum- maries from one language to another, which can contain errors from both summarization and translation processes. To address this issue, we propose ConvSumX, a cross-lingual con- versation summarization benchmark, through a new annotation schema that explicitly consid- ers source input context. ConvSumX consists of 2 sub-tasks under different real-world scenar- ios, with each covering 3 language directions. We conduct thorough analysis on ConvSumX and 3 widely-used manually annotated CLS corpora and empirically find that ConvSumX is more faithful towards input text. Addition- ally, based on the same intuition, we propose a 2-Step method, which takes both conversation and summary as input to simulate human anno- tation process. Experimental results show that 2-Step method surpasses strong baselines on ConvSumX under both automatic and human evaluation. Analysis shows that both source in- put text and summary are crucial for modeling cross-lingual summaries. 1 Introduction With the advance in deep learning and pre-trained language models (PLMs) (Devlin et al., 2019; Lewis et al., 2020; Raffel et al., 2020), much recent progress has been made in text summarization (Liu and Lapata, 2019; Zhong et al., 2022a; Chen et al., 2022a). However, most work focuses on English (En) data (Zhong et al., 2021; Gliwa et al., 2019; Chen et al., 2021), which does not consider cross- lingual sources for summarization (Wang et al., 2022b). To address this limitation, cross-lingual summarization (CLS) aims to generate summaries in a target language given texts from a source lan- guage (Zhu et al., 2019), which has shown values ∗Yue Zhang is the corresponding author. Fifty Five Percent Of Americans Think Pot Should Be Legal; Ex- Viking Kluwe Slams “Bigot” And “Cowards” Fifty Five Percent Of Americans Think Pot Should Be Legal; Ex- Viking Kluwe Slams “Bigot” And “Cowards” (a) Monolingual Summarization COSTELLO: Chris Kluwe certainly has a way of words. His article, “I was an NFL player until I was fired by two cowards and a bigot” … Monolingual Summarize Cross-lingual Summarize (b) Translation rich context information guiding information Translate 55%!"#$%&'()*+,-. /0123456789:;< 7=>; (c) Our CLS Annotation COSTELLO: Chris Kluwe certainly has a way of words. His article, “I was an NFL player until I was fired by two cowards and a bigot”, has now gone viral and it’s “Dead Spin’s” most popular post ever. The article alleges Kluwe was fired because of his outspoken … (Fifty Five Percent Of Americans Think Pot Should Be Legal; …

Content truncated to 500 words