ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7716 - 7729 December 7-11, 2022 ©2022 Association for Computational Linguistics CLIDSUM: A Benchmark Dataset for Cross-Lingual Dialogue Summarization Jiaan Wang1,2∗, Fandong Meng1†, Ziyao Lu1, Duo Zheng4 Zhixu Li3†, Jianfeng Qu2 and Jie Zhou1 1Pattern Recognition Center, WeChat AI, Tencent Inc, China 2School of Computer Science and Technology, Soochow University, Suzhou, China 3Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, China 4Beijing University of Posts and Telecommunications, Beijing, China jawang1@stu.suda.edu.cn, fandongmeng@tencent.com, zhixuli@fudan.edu.cn Abstract We present CLIDSUM, a benchmark dataset towards building cross-lingual summarization systems on dialogue documents. It consists of 67k+ dialogue documents and 112k+ anno- tated summaries in different target languages. Based on the proposed CLIDSUM, we intro- duce two benchmark settings for supervised and semi-supervised scenarios, respectively. We then build various baseline systems in dif- ferent paradigms (pipeline and end-to-end) and conduct extensive experiments on CLIDSUM to provide deeper analyses. Furthermore, we propose mDIALBART which extends mBART via further pre-training, where the multiple ob- jectives help the pre-trained model capture the structural characteristics as well as key con- tent in dialogues and the transformation from source to the target language. Experimental results show the superiority of mDIALBART, as an end-to-end model, outperforms strong pipeline models on CLIDSUM. Finally, we dis- cuss specific challenges that current approaches faced with this task and give multiple promis- ing directions for future research. We have released the dataset and code at https:// github.com/krystalan/ClidSum. 1 Introduction Cross-Lingual Summarization (XLS) aims at gen- erating a summary in the target language from a given document in a different source language. Existing XLS datasets mainly focus on single- participant documents, such as news reports (Zhu et al., 2019; Nguyen and Daumé III, 2019; Bai et al., 2021), how-to guides (Ladhak et al., 2020) and encyclopedia articles (Perez-Beltrachini and Lapata, 2021). Many efforts have been devoted to improving XLS for single-participant documents and achieved great success (Yao et al., 2015; Shen ∗Work was done when Jiaan Wang was interning at Pat- tern Recognition Center, WeChat AI, Tencent Inc, China. †Corresponding authors. Jack: Cocktails later? May: YES!!! May: You read my mind... Jack: Possibly a little tightly strung today? May: Sigh... without question. Jack: Thought so. German Summary: Jack und May werden später cocktails trinken. Chinese Summary: 杰克和梅一会儿要喝鸡尾酒。 Figure 1: An example of cross-lingual dialogue sum- marization. Blue and orange sentences are the German and Chinese summaries of the given English dialogue, respectively. et al., 2018; Ouyang et al., 2019; Cao et al., 2020; Zhu et al., 2020; Maurya et al., 2021; Chi et al., 2021; Liang et al., 2022; Wang et al., 2022). Different from single-participant documents, di- alogue is a discourse produced by more than one person (Haviland, 1990). The multi-participant dialogue documents, which record the communi- cations between human and human/machine, have attracted wide research attention due to their key role in daily interpersonal interaction (Zhang et al., 2020c). Meanwhile, the globalization progress has prompted conversations among …

Content truncated to 500 words