Cross-Lingual Abstractive Summarization with Limited Parallel Resources

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6910–6924 August 1–6, 2021. ©2021 Association for Computational Linguistics 6910 Cross-Lingual Abstractive Summarization with Limited Parallel Resources Yu Bai, Yang Gao, Heyan Huang∗ School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China Southeast Academy of Information Technology, Fujian, China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing, China {yubai,gyang,hhy63}@bit.edu.cn Abstract Parallel cross-lingual summarization data is scarce, requiring models to better use the lim- ited available cross-lingual resources. Exist- ing methods to do so often adopt sequence- to-sequence networks with multi-task frame- works. Such approaches apply multiple de- coders, each of which is utilized for a specific task. However, these independent decoders share no parameters, hence fail to capture the relationships between the discrete phrases of summaries in different languages, breaking the connections in order to transfer the knowl- edge of the high-resource languages to low- resource languages. To bridge these connec- tions, we propose a novel Multi-Task frame- work for Cross-Lingual Abstractive Summa- rization (MCLAS) in a low-resource setting. Employing one unified decoder to generate the sequential concatenation of monolingual and cross-lingual summaries, MCLAS makes the monolingual summarization task a prerequi- site of the cross-lingual summarization (CLS) task. In this way, the shared decoder learns in- teractions involving alignments and summary patterns across languages, which encourages attaining knowledge transfer. Experiments on two CLS datasets demonstrate that our model significantly outperforms three baseline mod- els in both low-resource and full-dataset sce- narios. Moreover, in-depth analysis on the generated summaries and attention heads ver- ifies that interactions are learned well using MCLAS, which benefits the CLS task under limited parallel resources. 1 Introduction Cross-lingual summarization (CLS) helps people efficiently grasp salient information from articles in a foreign language. Neural approaches to CLS require large scale datasets containing millions of cross-lingual document-summary pairs (Zhu et al., ∗Corresponding author. Figure 1: An example of the alignments across sum- maries in different languages. Each color represents phrases with one specific meaning. 2019; Cao et al., 2020; Zhu et al., 2020). How- ever, two challenges arise with these approaches: 1) most languages are low-resource, thereby lack- ing document-summary paired data; 2) large par- allel datasets across different languages for neural- based CLS are rare and expensive, especially under the current trend of neural networks. Therefore, a low-resource setting is more realistic, and chal- lenging, one for cross-lingual summarization. To our best knowledge, cross-lingual summarization under low-resource settings has not been well in- vestigated and explored. Therefore, in this paper, we will develop a new model for cross-lingual ab- stractive summarization under limited supervision. For low-resource settings, multi-task learning has been shown to be an effective method since it can borrow useful knowledge from other relevant tasks to use in the target task (Yan et al., 2015; Wang et al., 2020; Motiian et al., 2017). Cross- lingual summarization can be viewed as the combi- …

Content truncated to 500 words