Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1453–1463 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1453 Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization Zhengyuan Liu, Ke Shi, Nancy F. Chen Institute for Infocomm Research, A*STAR, Singapore {liu zhengyuan,shi ke,nfychen}@i2r.a-star.edu.sg Abstract Much progress has been made in text sum- marization, fueled by neural architectures us- ing large-scale training corpora. However, in the news domain, neural models easily over- fit by leveraging position-related features due to the prevalence of the inverted pyramid writ- ing style. In addition, there is an unmet need to generate a variety of summaries for differ- ent users. In this paper, we propose a neu- ral framework that can flexibly control sum- mary generation by introducing a set of sub- aspect functions (i.e. importance, diversity, po- sition). These sub-aspect functions are regu- lated by a set of control codes to decide which sub-aspect to focus on during summary gen- eration. We demonstrate that extracted sum- maries with minimal position bias is compa- rable with those generated by standard mod- els that take advantage of position preference. We also show that news summaries generated with a focus on diversity can be more preferred by human raters. These results suggest that a more flexible neural summarization frame- work providing more control options could be desirable in tailoring to different user prefer- ences, which is useful since it is often imprac- tical to articulate such preferences for different applications a priori. 1 Introduction Text summarization targets to automatically gen- erate a shorter version of the source content while retaining the most important information. As a straightforward and effective method, extractive summarization creates a summary by selecting and subsequently concatenating the most salient semantic units in a document. Recently, neural approaches, often trained in an end-to-end man- ner, have achieved favorable improvements on var- ious large-scale benchmarks (Nallapati et al., 2017; Narayan et al., 2018a; Liu and Lapata, 2019). Despite renewed interest and avid development in extractive summarization, there are still long- standing, unresolved challenges. One major prob- lem is position bias, which is especially common in the news domain, where the majority of research in summarization is studied. In many news articles, sentences appearing earlier tend to be more impor- tant for summarization tasks (Hong and Nenkova, 2014), and this preference is reflected in reference summaries of public datasets. However, while this tendency is common due to the classic text- book writing style of the “inverted pyramid” (Scan- lan, 1999), news articles can be presented in vari- ous ways. Other journalism writing styles include anecdotal lead, question-and-answer format, and chronological organization (Stovall, 1985). There- fore, salient information could also be scattered across the entire article, instead of being concen- trated in the first few sentences, depending on the chosen writing style of the journalist. As the “inverted pyramid” style is widespread in news articles (Kryscinski et al., 2019), neural models would easily overfit on position-related fea- tures in extractive summarization tasks because …

Content truncated to 500 words