A Variational Hierarchical Model for Neural Cross-Lingual Summarization

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2088 - 2099 May 22-27, 2022 c⃝2022 Association for Computational Linguistics A Variational Hierarchical Model for Neural Cross-Lingual Summarization Yunlong Liang1∗, Fandong Meng2, Chulun Zhou2,3, Jinan Xu1†, Yufeng Chen1, Jinsong Su3 and Jie Zhou2 1Beijing Key Lab of Traffic Data Analysis and Mining, Beijing Jiaotong University, China 2Pattern Recognition Center, WeChat AI, Tencent Inc, China 3School of Informatics, Xiamen University {yunlongliang,jaxu,chenyf}@bjtu.edu.cn clzhou@stu.xmu.edu.cn jssu@xmu.edu.cn {fandongmeng,withtomzhou}@tencent.com Abstract The goal of the cross-lingual summarization (CLS) is to convert a document in one language (e.g., English) to a summary in another one (e.g., Chinese). Essentially, the CLS task is the combination of machine translation (MT) and monolingual summarization (MS), and thus there exists the hierarchical relationship be- tween MT&MS and CLS. Existing studies on CLS mainly focus on utilizing pipeline meth- ods or jointly training an end-to-end model through an auxiliary MT or MS objective. How- ever, it is very challenging for the model to di- rectly conduct CLS as it requires both the abili- ties to translate and summarize. To address this issue, we propose a hierarchical model for the CLS task, based on the conditional variational auto-encoder. The hierarchical model contains two kinds of latent variables at the local and global levels, respectively. At the local level, there are two latent variables, one for transla- tion and the other for summarization. As for the global level, there is another latent variable for cross-lingual summarization conditioned on the two local-level variables. Experiments on two language directions (English⇔Chinese) verify the effectiveness and superiority of the pro- posed approach. In addition, we show that our model is able to generate better cross-lingual summaries than comparison models in the few- shot setting. 1 Introduction The cross-lingual summarization (CLS) aims to summarize a document in source language (e.g., English) into a different language (e.g., Chinese), which can be seen as a combination of machine translation (MT) and monolingual summarization (MS) to some extent (Or˘asan and Chiorean, 2008; Zhu et al., 2019). The CLS can help people ef- fectively master the core points of an article in a ∗Work was done when Liang and Zhou were interning at Pattern Recognition Center, WeChat AI, Tencent Inc, China. †Jinan Xu is the corresponding author. foreign language. Under the background of glob- alization, it becomes more important and is now coming into widespread use in real life. Many researches have been devoted to deal- ing with this task. To our knowledge, they mainly fall into two categories, i.e., pipeline and end-to-end learning methods. (i) The first cate- gory is pipeline-based, adopting either translation- summarization (Leuski et al., 2003; Ouyang et al., 2019) or summarization-translation (Wan et al., 2010; Or˘asan and Chiorean, 2008) paradigm. Al- though being intuitive and straightforward, they generally suffer from error propagation problem. (ii) The second category aims to train an end- to-end model for CLS (Zhu et al., 2019, 2020). For instance, Zhu et al. (2020) focus on using a …

Content truncated to 500 words