Jointly Learning Salience and Redundancy by Adaptive Sentence Reranking for Extractive Summarization

Jointly Learning Salience and Redundancy by Adaptive Sentence Reranking for Extractive Summarization Ximing Zhang, Ruifang Liu∗ Beijing University of Posts and Telecommunications, Beijing 100876, China ximingzhang@bupt.edu.cn, lrf@bupt.edu.cn Abstract Extractive text summarization seeks to extract indicative sentences from a source document and assemble them to form a summary. Selecting salient but not redundant sentences has always been the main challenge. Unlike the previous two-stage strategies, this paper presents a unified end-to-end model, learning to rerank the sentences by modeling salience and redundancy simul- taneously. Through this ranking mechanism, our method can improve the quality of the overall candidate summary by giving higher scores to sentences that can bring more novel informa- tion. We first design a summary-level measure to evaluate the cumulating gain of each candidate summaries. Then we propose an adaptive training objective to rerank the sentences aiming at obtaining a summary with a high summary-level score. The experimental results and evalua- tion show that our method outperforms the strong baselines on three datasets and further boosts the quality of candidate summaries, which intensely indicate the effectiveness of the proposed framework. 1 Introduction Extractive summarization aims to create a summary by identifying and concatenating the most important sentences in a document (Nallapati et al., 2016; Zhou et al., 2018; Dong et al., 2018; Liu and Lapata, 2019; Zhang et al., 2019). To choose an appropriate sentence from the document, we need to consider two aspects: salience, which represents how much information the sentence carries; and redundancy, which represents how much information in the sentence is already included in the previously selected sentences(Peyrard, 2019). The former focuses on sentence-level importance, while the latter considers the relationship between sentences at the summary-level. The main challenge of extractive summarization is how to combine salience and redundancy simulta- neously. Most previous methods (Cheng and Lapata, 2016; Kedzie et al., 2018) only consider salience at the sentence-level, where they usually model sentence-selecting as a sequence labeling task. Several approaches for modeling redundancy between selected sentences are generally classified into two types: heuristics-based and model-based approaches(Ren et al., 2016). The former such as Trigram Blocking (Liu and Lapata, 2019) is not adaptive since they usually apply the same rule to all the documents, which results in limited effects on a few specific datasets. The latter depends heavily on feature engineering or a neural post-processing module to model redundancy. (Bi et al., 2020) extracts Ngram-matching and semantic-matching features to indicate the redundancy of a candidate sentence. (Zhong et al., 2020) pro- poses a two-step pipeline that first scores salience, then learns to balance salience and redundancy as a text-matching task. The critical drawbacks are error propagation and high computation cost. Compared to these models, we aim to propose an efficient and unified end-to-end model to jointly learn salience and redundancy without extra post-processing modules. ∗Corresponding author. ©2021 China National Conference on Computational Linguistics Published under Creative Commons Attribution 4.0 International License CCL 2021 Proceedings of the 20th China National Conference on Computational Linguistics, pages 952-963, Hohhot, China, …

Content truncated to 500 words