Unsupervised Abstractive Summarization of Bengali Text Documents

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2612–2619 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2612 Unsupervised Abstractive Summarization of Bengali Text Documents Radia Rayan Chowdhury∗ Ahsanullah Univ of Science & Tech Dhaka, Bangladesh radiarayan.rrc@gmail.com Mir Tafseer Nayeem∗ Ahsanullah Univ of Science & Tech Dhaka, Bangladesh mir.nayeem@alumni.uleth.ca Tahsin Tasnim Mim Ahsanullah Univ of Science & Tech Dhaka, Bangladesh tahsintasnimmim@gmail.com Md. Saifur Rahman Chowdhury Ahsanullah Univ of Science & Tech Dhaka, Bangladesh saif.chowdhury1997@gmail.com Taufiqul Jannat Ahsanullah Univ of Science & Tech Dhaka, Bangladesh taufiquljannat@gmail.com Abstract Abstractive summarization systems gener- ally rely on large collections of document- summary pairs. However, the performance of abstractive systems remains a challenge due to the unavailability of the parallel data for low-resource languages like Bengali. To over- come this problem, we propose a graph-based unsupervised abstractive summarization sys- tem in the single-document setting for Bengali text documents, which requires only a Part- Of-Speech (POS) tagger and a pre-trained lan- guage model trained on Bengali texts. We also provide a human-annotated dataset with document-summary pairs to evaluate our ab- stractive model and to support the comparison of future abstractive summarization systems of the Bengali Language. We conduct exper- iments on this dataset and compare our system with several well-established unsupervised ex- tractive summarization systems. Our unsuper- vised abstractive summarization model outper- forms the baselines without being exposed to any human-annotated reference summaries.1 1 Introduction The process of shortening a large text document with the most relevant information of the source is known as automatic text summarization. A good summary should be coherent, non-redundant, and grammatically readable while retaining the original document’s most important contents (Nenkova and McKeown, 2012; Nayeem et al., 2018). There are ∗Equal contribution. Listed by alphabetical order. at1We make our code & dataset publicly available https://github.com/tafseer-nayeem/ BengaliSummarization for reproduciblity. two types of summarizations: extractive and ab- stractive. Extractive summarization is about rank- ing important sentences from the original text. The abstractive method generates human-like sentences using natural language generation techniques. Tra- ditionally used abstractive techniques are sentence compression, syntactic reorganization, sentence fu- sion, and lexical paraphrasing (Lin and Ng, 2019). Compared to extractive, abstractive summary gen- eration is indeed a challenging task. A cluster of sentences uses multi-sentence com- pression (MSC) to summarize into one single sentence originally called sentence fusion (Barzi- lay and McKeown, 2005; Nayeem and Chali, 2017b). The success of neural sequence-to- sequence (seq2seq) models with attention (Bah- danau et al., 2015; Luong et al., 2015) provides an effective way for text generation which has been extensively applied in the case of abstractive sum- marization of English language documents (Rush et al., 2015; Chopra et al., 2016; Nallapati et al., 2016; Miao and Blunsom, 2016; Paulus et al., 2018; Nayeem et al., 2019). These models are usually trained with lots of gold summaries, but there is no large-scale human-annotated abstractive sum- maries available for low-resource language like Bengali. In contrast, the unsupervised approach reduces the human effort and cost …

Content truncated to 500 words