Jointly Learning to Align and Summarize for Neural Cross-Lingual Summarization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6220–6231 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6220 Jointly Learning to Align and Summarize for Neural Cross-Lingual Summarization Yue Cao, Hui Liu, Xiaojun Wan Wangxuan Institute of Computer Technology, Peking University Center for Data Science, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University {yuecao,xinkeliuhui,wanxiaojun}@pku.edu.cn Abstract Cross-lingual summarization is the task of gen- erating a summary in one language given a text in a different language. Previous works on cross-lingual summarization mainly focus on using pipeline methods or training an end- to-end model using the translated parallel data. However, it is a big challenge for the model to directly learn cross-lingual summarization as it requires learning to understand different languages and learning how to summarize at the same time. In this paper, we propose to ease the cross-lingual summarization training by jointly learning to align and summarize. We design relevant loss functions to train this framework and propose several methods to en- hance the isomorphism and cross-lingual trans- fer between languages. Experimental results show that our model can outperform compet- itive models in most cases. In addition, we show that our model even has the ability to generate cross-lingual summaries without ac- cess to any cross-lingual corpus. 1 Introduction Neural abstractive summarization has witnessed rapid growth in recent years. Variants of sequence- to-sequence models have shown to obtain promis- ing results on English (See et al., 2017) or Chinese summarization datasets. However, Cross-lingual summarization, which aims at generating a sum- mary in one language from input text in a different language, has been rarely studied because of the lack of parallel corpora. tiveEarly researches on cross-lingual abstrac- summarization are mainly based on the summarization-translation or translation- summarization pipeline paradigm and adopt different strategies to incorporate bilingual features (Leuski et al., 2003; Orasan and Chiorean, 2008; Wan et al., 2010; Wan, 2011) into the pipeline model. Recently, Shen et al. (2018) first propose a neu- ral cross-lingual summarization system based on a large-scale corpus. They first translate the texts automatically from the source language into the target language and then use the teacher-student framework to train a cross-lingual summarization model. Duan et al. (2019) further improve this teacher-student framework by using genuine sum- maries paired with the translated pseudo source sentences to train the cross-lingual summarization model. Zhu et al. (2019) propose a multi-task learn- ing framework to train a neural cross-lingual sum- marization model. Cross-lingual summarization is a challenging task as it requires learning to understand different languages and learning how to summarize at the same time. It would be difficult for the model to directly learn cross-lingual summarization. In this paper, we explore this question: can we ease the training and enhance the cross-lingual summariza- tion by establishing alignment of context represen- tations between two languages? Learning cross-lingual representations has been proven a beneficial method for cross-lingual trans- fer for some downstream tasks (Klementiev et al., 2012; Artetxe et al., …

Content truncated to 500 words