Deep Recurrent Generative Decoder for Abstractive Text Summarization

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2091–2100 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Deep Recurrent Generative Decoder for Abstractive Text Summarization∗ Piji Li† Wai Lam† Lidong Bing‡ Zihao Wang† †Key Laboratory on High Confidence Software Technologies (Sub-Lab, CUHK), Ministry of Education, China †Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong ‡AI Lab, Tencent Inc., Shenzhen, China †{pjli, wlam, zhwang}@se.cuhk.edu.hk, ‡lyndonbing@tencent.com Abstract We propose a new framework for ab- stractive text summarization based on a sequence-to-sequence oriented encoder- decoder model equipped with a deep re- current generative decoder (DRGN). La- tent structure information implied in the target summaries is learned based on a re- current latent random model for improv- ing the summarization quality. Neural variational inference is employed to ad- dress the intractable posterior inference for the recurrent latent variables. Abstractive summaries are generated based on both the generative latent variables and the dis- criminative deterministic states. Extensive experiments on some benchmark datasets in different languages show that DRGN achieves improvements over the state-of- the-art methods. 1 Introduction Automatic summarization is the process of auto- matically generating a summary that retains the most important content of the original text doc- ument (Edmundson, 1969; Luhn, 1958; Nenkova and McKeown, 2012). Different from the common extraction-based and compression-based methods, abstraction-based methods aim at constructing new sentences as summaries, thus they require a deeper understanding of the text and the capabil- ity of generating new sentences, which provide an obvious advantage in improving the focus of a summary, reducing the redundancy, and keeping a good compression rate (Bing et al., 2015; Rush et al., 2015; Nallapati et al., 2016). ∗The work described in this paper is supported by a grant from the Grant Council of the Hong Kong Special Adminis- trative Region, China (Project Code: 14203414). Apple sues Qualcomm for nearly $1 billion Twitter fixes botched @POTUS account transfer Track Trump’s 100-day promises, Silicon Valley-style The emergence of the ‘cyber cold war’ Tesla Autopilot not defective in fatal crash Twitter mostly meets modest diversity goals Uber to pay $20 million for misleading drivers top stories_ Figure 1: Headlines of the top stories from the channel “Technology” of CNN. Some previous research works show that human-written summaries are more abstractive (Jing and McKeown, 2000). Moreover, our in- vestigation reveals that people may naturally fol- low some inherent structures when they write the abstractive summaries. To illustrate this observa- tion, we show some examples in Figure 1, which are some top story summaries or headlines from the channel “Technology” of CNN. After analyz- ing the summaries carefully, we can find some common structures from them, such as “What”, “What-Happened” , “Who Action What”, etc. For example, the summary “Apple sues Qual- comm for nearly $1 billion” can be structural- ized as “Who (Apple) Action (sues) What (Qual- comm)”. Similarly, the summaries “[Twitter] [fixes] [botched @POTUS account transfer]”, “[Uber] [to pay] [$20 million] for misleading drivers”, and “[Bipartisan bill] aims …

Content truncated to 500 words