Zero-Shot Cross-Lingual Abstractive Sentence Summarization through Teaching Generation and Attention

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3162–3172 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3162 Zero-Shot Cross-Lingual Abstractive Sentence Summarization through Teaching Generation and Attention Xiangyu Duan1,2∗, Mingming Yin1∗, Min Zhang1,2, Boxing Chen3, Weihua Luo3 1 Institute of Artificial Intelligence, Soochow University, Suzhou, China 2 School of Computer Science and Technology, Soochow University, Suzhou, China 3 Alibaba DAMO Academy, Hangzhou, China xiangyuduan@suda.edu.cn; mmyin@stu.suda.edu.cn; minzhang@suda.edu.cn {boxing.cbx,weihua.luowh}@alibaba-inc.com Abstract Abstractive Sentence Summarization (AS- SUM) targets at grasping the core idea of the source sentence and presenting it as the sum- mary. It is extensively studied using statis- tical models or neural models based on the large-scale monolingual source-summary par- allel corpus. But there is no cross-lingual parallel corpus, whose source sentence lan- guage is different to the summary language, to directly train a cross-lingual ASSUM sys- tem. We propose to solve this zero-shot prob- lem by using resource-rich monolingual AS- SUM system to teach zero-shot cross-lingual ASSUM system on both summary word gen- eration and attention. This teaching pro- cess is along with a back-translation pro- cess which simulates source-summary pairs. Experiments on cross-lingual ASSUM task show that our proposed method is significantly better than pipeline baselines and previous works, and greatly enhances the cross-lingual performances closer to the monolingual per- formances. We release the code and data at https://github.com/KelleyYin/ Cross-lingual-Summarization. 1 Introduction Abstractive Sentence Summarization (ASSUM) is a task of condensing the source sentences into the summaries based on the core meaning of the source sentences. ASSUM provides quick ac- cess to the important content of the source sen- tences through the informative re-written sum- maries. Major ASSUM explorations are monolin- gual based. There is an urgent demand of cross- lingual ASSUM which produces summaries for people who do not speak the language the same to the source language. Unlike the monolingual ASSUM receiving ex- tensive studies that are based on the large-scale ∗ Equal contribution. monolingual ASSUM corpus, the cross-lingual ASSUM is seldom explored due to the lack of training corpus. This zero-shot challenge drives the cross-lingual ASSUM to resort to two existing independent techniques, i.e., the monolingual AS- SUM and the bilingual translation. The both tech- niques should be leveraged together to overcome the difficulty of data scarcity in the cross-lingual ASSUM. Regarding the techniques of the monolingual ASSUM, neural methods become dominant in this area since the creation of the large-scale ASSUM corpus (Rush et al., 2015; Nallapati et al., 2016; Hu et al., 2015). The corpus consists of huge num- ber of source-summary pairs, and neural methods model these pairs as as a sequence-to-sequence task by encoding the source sentence into vector- ized information and decoding it into the abstrac- tive summary. Regarding the techniques of the bilingual trans- lation, recent years witnessed the method tran- sition from statistical machine translation (SMT) (Koehn et al., 2003) to neural machine translation (NMT). NMT employs the sequence-to-sequence architecture with various implementations such as RNN-based (Sutskever et al., …

Content truncated to 500 words