Systematically Exploring Redundancy Reduction in Summarizing Long Documents

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 516–528 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 516 Systematically Exploring Redundancy Reduction in Summarizing Long Documents Wen Xiao and Giuseppe Carenini Department of Computer Science University of British Columbia Vancouver, BC, Canada, V6T 1Z4 {xiaowen3, carenini}@cs.ubc.ca Abstract Our analysis of large summarization datasets indicates that redundancy is a very serious problem when summarizing long documents. Yet, redundancy reduction has not been thor- oughly investigated in neural summarization. In this work, we systematically explore and compare different ways to deal with redun- dancy when summarizing long documents. Specifically, we organize the existing meth- ods into categories based on when and how the redundancy is considered. Then, in the context of these categories, we propose three additional methods balancing non-redundancy and importance in a general and flexible way. In a series of experiments, we show that our proposed methods achieve the state-of-the-art with respect to ROUGE scores on two scien- tific paper datasets, Pubmed and arXiv, while reducing redundancy significantly. 1 1 Introduction Summarization is the task of shortening a given document(s) while maintaining the most important information. In general, a good summarizer should generate a summary that is syntactically accurate, semantically correct, coherent, and non-redundant (Saggion and Poibeau, 2013). While extractive methods tend to have better performance on the first two aspects, they are typically less coherent and more redundant than abstractive ones, where new sentences are often generated by sentence fu- sion and compression, which helps detecting and removing redundancy (Lebanoff et al., 2019). Al- though eliminating redundancy has been initially and more intensely studied in the field of multi- document summarization (Lloret and Sanz, 2013), because important sentences selected from mul- tiple documents (about the same topic) are more 1Our code can be found here - http://www.cs. ubc.ca/cs-research/lci/research-groups/ natural-language-processing/ likely to be redundant than sentences from the same document, generating a non-redundant summary should still be one of the goals for single document summarization (Lin et al., 2009). Generally speaking, there is a trade-off between importance and diversity (non-redundancy) (Jung et al., 2019), which is reflected in the two phases, sentence scoring and sentence selection (Zhou et al., 2018) in which extractive summarization task can be naturally decomposed. The former typ- ically scores sentences based on importance, while the latter selects sentences based on their scores, but also possibly taking other factors (including redundancy) into account. Traditionally, in non-neural approaches the trade- off between importance and redundancy has been carefully considered, with sentence selection pick- ing sentences by optimizing an objective function that balances the two aspects (Carbonell and Gold- stein, 1998; Ren et al., 2016). In contrast, more recent works on neural extractive summarization models has so far over-emphasized sentence impor- tance and the corresponding scoring phase, while paying little attention to how to reduce redundancy in the selection phase, where they simply apply a greedy algorithm to …

Content truncated to 500 words