Global Optimization under Length Constraint for Neural Text Summarization

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1039–1048 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1039 Global Optimization under Length Constraint for Neural Text Summarization Takuya Makino and Tomoya Iwakura Fujitsu Laboratories, Ltd. {makino.takuya,iwakura.tomoya}@fujitsu.com Hiroya Takamura AIST takamura.hiroya@aist.go.jp Manabu Okumura Tokyo Institute of Technology oku@pi.titech.ac.jp Abstract We propose a global optimization method under length constraint (GOLC) for neural text summarization models. GOLC increases the probabilities of generating summaries that have high evaluation scores, ROUGE in this paper, within a desired length. We com- pared GOLC with two optimization meth- ods, a maximum log-likelihood and a mini- mum risk training, on CNN/Daily Mail and a Japanese single document summarization data set of The Mainichi Shimbun Newspapers. The experimental results show that a state-of- the-art neural summarization model optimized with GOLC generates fewer overlength sum- maries while maintaining the fastest process- ing speed; only 6.70% overlength summaries on CNN/Daily and 7.8% on long summary of Mainichi, compared to the approximately 20% to 50% on CNN/Daily Mail and 10% to 30% on Mainichi with the other optimization methods. We also demonstrate the importance of the generation of in-length summaries for post-editing with the dataset Mainich that is created with strict length constraints. The ex- perimental results show approximately 30% to 40% improved post-editing time by use of in- length summaries. 1 Introduction Automatic text summarization aims at generating a short and coherent summary of a given text. In text summarization, while the generated sum- maries should contain the important content of the input text, their lengths should also be controlled, e.g., the summary should be as long as the width of target devices such as smart-phones and digital signage. Therefore, editors have to summarize a source text under a length constraint by reordering and paraphrasing. For summarization, both extractive and abstrac- tive methods have been widely studied. Extractive methods are based on selection of sentences from source texts without using reordering or para- phrasing. In contrast, abstractive methods gener- ate summaries as new sentences. Therefore, ab- stractive methods can rely on the reordering and paraphrasing required for summary and title gen- eration. However, most abstractive summariza- tion methods are not able to control the summary length. To this problem, Kikuchi et al. (2016) and Liu et al. (2018) proposed abstractive summariza- tion models with a capability of summary length control. One is an LSTM based summarization model, and the other is a CNN based one. They proposed to enforce the desired length in the de- coding of training and generation. Their models, however, leave much room for improvement, at least with regard to two aspects. One aspect is that the summarization performance is still worse than other state-of-the-art models. The other is that their models sometimes fail to control the output length. In this paper, we address these two issues by in- corporating global training based on a minimum risk training (MRT) under the length constraint. MRT (Och, 2003) is used …

Content truncated to 500 words