Summary Refinement through Denoising

Proceedings of Recent Advances in Natural Language Processing, pages 837–843, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_097 837 Summary Refinement through Denoising Nikola I. Nikolov, Alessandro Calmanovici, Richard H.R. Hahnloser Institute of Neuroinformatics, University of Z¨urich and ETH Z¨urich, Switzerland {niniko, dcalma, rich}@ini.ethz.ch Abstract We propose a simple method for post- processing the outputs of a text summa- rization system in order to refine its over- all quality. Our approach is to train text- to-text rewriting models to correct infor- mation redundancy errors that may arise during summarization. We train on syn- thetically generated noisy summaries, test- ing three different types of noise that in- troduce out-of-context information within each summary. When applied on top of extractive and abstractive summarization baselines, our summary denoising models yield metric improvements while reducing redundancy.1 1 Introduction Text summarization aims to produce a shorter, in- formative version of an input text. While ex- tractive summarization only selects important sen- tences from the input, abstractive summariza- tion generates content without explicitly re-using whole sentences (Nenkova et al., 2011). In recent years, a number of successful approaches have been proposed for both extractive (Nallapati et al., 2017; Narayan et al., 2018) and abstractive (Chen and Bansal, 2018; Gehrmann et al., 2018) summa- rization paradigms. Despite these successes, many state-of-the-art systems remain plagued by overly high output redundancy (See et al. (2017); see Fig- ure 3), which we set out to reduce. In this paper, we propose a simple method (Fig- ure 1, Section 3) for post-processing the outputs of a text summarization system in order to im- prove their overall quality. Our approach is to train dedicated text-to-text rewriting models to correct 1Code available at https://github.com/ ninikolov/summary-denoising. Ground Truth Summary Noisy Summary 1. Synthesize noisy summaries Denoising Model Noise Model Clean Summary Noisy Summary 2. Train/apply summary denoising model Figure 1: Overview of our approach to summary denoising. We alter ground truth summaries to generate a noisy dataset, on which we train denoising models to restore the original summaries. errors that may arise during summarization, fo- cusing specifically on reducing information redun- dancy within each individual summary. To achieve this, we synthesize from clean summaries noisy summaries that contain diverse information redun- dancy errors, such as sentence repetition and out- of-context information (Section 3.2). In our experiments (Section 5), we show that denoising yields metric improvements and reduces redundancy when applied on top of several extrac- tive and abstractive baselines. The generality of our method makes it a useful post-processing step applicable to any summarization system, that stan- dardizes the summaries and improves their over- all quality, ensuring fewer redundancies across the text. 2 Background Post-processing of noisy human or machine- generated text is a topic that has recently been gathering interest. Automatic error correction (Rozovskaya and Roth, 2016; Xie et al., 2018) aims to improve the grammar or spelling of a text. In machine translation, automatic post edit- ing of translated outputs (Chatterjee et al., 2018) is commonly used to further improve the translation quality, standardise …

Content truncated to 500 words