BiSET: Bi-directional Selective Encoding with Template for Abstractive Summarization

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2153–2162 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2153 BiSET: Bi-directional Selective Encoding with Template for Abstractive Summarization Kai Wang Sun Yat-sen University wangk73@mail2.sysu.edu.cn Xiaojun Quan∗ Sun Yat-sen University quanxj3@mail.sysu.edu.cn Rui Wang Alibaba Inc. masi.wr@alibaba-inc.com Abstract The success of neural summarization models stems from the meticulous encodings of source articles. To overcome the impediments of lim- ited and sometimes noisy training data, one promising direction is to make better use of the available training data by applying filters during summarization. In this paper, we pro- pose a novel Bi-directional Selective Encoding with Template (BiSET) model, which lever- ages template discovered from training data to softly select key information from each source article to guide its summarization process. Ex- tensive experiments on a standard summariza- tion dataset were conducted and the results show that the template-equipped BiSET model manages to improve the summarization perfor- mance significantly with a new state of the art. 1 Introduction Abstractive summarization aims to shorten a source article or paragraph by rewriting while pre- serving the main idea. Due to the difficulties in rewriting long documents, a large body of research on this topic has focused on paragraph-level ar- ticle summarization. Among them, sequence-to- sequence models have become the mainstream and some have achieved state-of-the-art performance (Rush et al., 2015; Chopra et al., 2016; Nallap- ati et al., 2016). In general, the only available information for these models during decoding is simply the source article representations from the encoder and the generated words from the previ- ous time steps (Nallapati et al., 2016; Gu et al., 2016; Lin et al., 2018), while the previous words are also generated based on the article representa- tions. Since natural language text is complicated and verbose in nature, and training data is insuffi- cient in size to help the models distinguish impor- tant article information from noise, sequence-to- ∗Corresponding author. sequence models tend to deteriorate with the ac- cumulation of word generation, e.g., they generate irrelevant and repeated words frequently (Koehn and Knowles, 2017). Template-based summarization (Zhou and Hovy, 2004) is an effective approach to traditional abstractive summarization, in which a number of hard templates are manually created by domain experts, and key snippets are then extracted and populated into the templates to form the final summaries. The advantage of such approach is it can guarantee concise and coherent summaries in no need of any training data. However, it is unre- alistic to create all the templates manually since this work requires considerable domain knowl- edge and is also labor-intensive. Fortunately, the summaries of some specific training articles can provide similar guidance to the summarization as hard templates. Accordingly, these summaries are referred to as soft templates, or templates for simplicity, in this paper. Despite their potential in relieving the verbosity and insufficiency problems of natural language data, templates have not been exploited to full ad- vantage. For example, Cao et …

Content truncated to 500 words