Sliding Selector Network with Dynamic Memory for Extractive Summarization of Long Documents

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5881–5891 June 6–11, 2021. ©2021 Association for Computational Linguistics 5881 Sliding Selector Network with Dynamic Memory for Extractive Summarization of Long Documents Peng Cui and Le Hu School of Computer Science and Technology Harbin Institute of Technology, Harbin, China {pcui,lhu}@insun.hit.edu.cn Abstract Neural-based summarization models suffer from the length limitation of text encoder. Long documents have to been truncated be- fore they are sent to the model, which re- sults in huge loss of summary-relevant con- tents. To address this issue, we propose the sliding selector network with dynamic mem- ory for extractive summarization of long-form documents, which employs a sliding window to extract summary sentences segment by seg- ment. Moreover, we adopt memory mecha- nism to preserve and update the history in- formation dynamically, allowing the seman- tic flow across different windows. Experi- mental results on two large-scale datasets that consist of scientific papers demonstrate that our model substantially outperforms previous state-of-the-art models. Besides, we perform qualitative and quantitative investigations on how our model works and where the perfor- mance gain comes from.1 1 Introduction Text summarization is an important task of natural language processing which aims to distil salient contents from a textual document. Existing sum- marization models can be roughly classified into two categories, which are abstractive and extractive. Abstractive summarization usually adopts natural language generation technology to produce a word- by-word summary. In general, these approaches are flexible but may yield disfluent summaries (Liu and Lapata, 2019a). By comparison, extractive ap- proaches aim to select a subset of the sentences in the source document, thereby enjoying better fluency and efficiency (Cao et al., 2017). Although many summarization approaches have demonstrated their success on relatively short doc- uments, such as news articles, they usually fail 1Code will be released at https://github.com/ pcui-nlp/SSN_DM Paragraph 1: Medical tourism is illustrated as occurrence in which individuals travel abroad to receive healthcare services. It is a multi- billion dollar industry and countries like India, Thailand, Israel, Singapore, … Paragraph 2: The prime driving factors in medical tourism are increased medical costs, increased insurance premiums, increasing number of uninsured or partially insured individuals in developed countries, … …… Paragraph 5: It is generally presumed in marketing that products with similar characteristics will be equally preferred by the consumers, however, attributes, which make the product similar to other products, will not…. Figure 1: An example where a paragraph-by-paragraph extraction will produce an incoherent summary. to achieve desired performance when directly ap- plied in long-form documents, such as scientific papers. This inferior performance is partly due to the truncation operation, which inevitably leads to information loss, especially for extractive models because parts of gold sentences would be inacces- sible. In addition, the accurate modeling of long texts remains a challenge (Frermann and Klemen- tiev, 2019). A practical solution for this problem is to use a sliding window to process documents separately. This approach is used …

Content truncated to 500 words