Enriching and Controlling Global Semantics for Text Summarization

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9443–9456 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9443 Enriching and Controlling Global Semantics for Text Summarization Thong Nguyen1,3∗, Anh Tuan Luu2† , Truc Lu3, Tho Quan3 1VinAI Research, Vietnam 2Nanyang Technological University, Singapore 3Ho Chi Minh City University of Technology, Vietnam National University Ho Chi Minh City v.thongnt66@vinai.io, tuancs219@yahoo.com, qttho@hcmut.edu.vn Abstract Recently, Transformer-based models have been proven effective in the abstractive sum- marization task by creating fluent and informa- tive summaries. Nevertheless, these models still suffer from the short-range dependency problem, causing them to produce summaries that miss the key points of document. In this paper, we attempt to address this issue by in- troducing a neural topic model empowered with normalizing flow to capture the global se- mantics of the document, which are then inte- grated into the summarization model. In ad- dition, to avoid the overwhelming effect of global semantics on contextualized representa- tion, we introduce a mechanism to control the amount of global semantics supplied to the text generation module. Our method outperforms state-of-the-art summarization models on five common text summarization datasets, namely CNN/DailyMail, XSum, Reddit TIFU, arXiv, and PubMed. 1 Introduction Automatic text summarization corresponds to text understanding and text generation processes. In general, there are two main approaches to per- form this task. Extractive systems (Liu, 2019; Narayan et al., 2020; Zhang et al., 2019; Jia et al., 2020) highlight salient words or sentences from the source text and form the final summary by concate- nating them. On the other hand, abstractive meth- ods (See et al., 2017; Zhang et al., 2020; Zou et al., 2020) switch among generating new words, choos- ing phrases from the source document, and rephras- ing them. Abstractive summarization, which is the focus of this paper, is usually more advanced and closer to human-like interpretation. Recently, abstractive summarization studies (Lewis et al., 2019; Zhang et al., 2020; Chen and Yang, 2020) are dominated by Transformer-based ∗Work done during internship at VinAI Research †Corresponding Author DOCUMENT: While Richie Benaud rose from the suburbs to captain Australia, he will be remembered forever for his mastery of commentating. The champion leg spinner turned cricket commentating into an art form, earning him the title of ’the Voice of Cricket.’ His commentary was understated, measured and often extremely funny, and were perfectly timed. Scroll down for video. 84-year-old cricket commen- tator Richie Benaud has passed away after a battle with skin cancer . His sayings from the hundreds of Test and One Day cricket matches he commentated on across the world were often what fans remembered from important moments. His signature one liners soon dropped to a simple word. ’Mar- vellous...’ will forever be linked to the cricket legend. On commentating, Benaud said: ’My mantra is - put your brain into gear and if you can add to what’s on the screen then do it, otherwise shut up.’ He once described the scene on the field: ’From our broadcast box …

Content truncated to 500 words