Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses
8898Authors
nan
Abstract
AbstractSentence position is a strong feature for news summarization, since the lead often (but not always) summarizes the key points of the article. In this paper, we show that recent neural systems excessively exploit this trend, which although powerful for many inputs, is also detrimental when summarizing documents where important content should be extracted from later parts of the article. We propose two techniques to make systems sensitive to the importance of content in different parts of the article. The first technique employs ‘unbiased’ data; i.e., randomly shuffled sentences of the source document, to pretrain the model. The second technique uses an auxiliary ROUGE-based loss that encourages the model to distribute importance scores throughout a document by mimicking sentence-level ROUGE scores on the training data. We show that these techniques significantly improve the performance of a competitive reinforcement learning based extractive system, with the auxiliary loss being more powerful than pretraining.
Keywords
Full Content
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6019–6024, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6019 Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses Matt Grenander∗, Yue Dong∗ McGill University / MILA {matthew.grenander, yue.dong2}@mail.mcgill.ca Jackie C. K. Cheung McGill University / MILA jcheung@cs.mcgill.ca Annie Louis University of Edinburgh Alan Turing Institute alouis@inf.ed.ac.uk Abstract Sentence position is a strong feature for news summarization, since the lead often (but not al- ways) summarizes the key points of the article. In this paper, we show that recent neural sys- tems excessively exploit this trend, which al- though powerful for many inputs, is also detri- mental when summarizing documents where important content should be extracted from later parts of the article. We propose two tech- niques to make systems sensitive to the impor- tance of content in different parts of the arti- cle. The first technique employs ‘unbiased’ data; i.e., randomly shuffled sentences of the source document, to pretrain the model. The second technique uses an auxiliary ROUGE- based loss that encourages the model to dis- tribute importance scores throughout a doc- ument by mimicking sentence-level ROUGE scores on the training data. We show that these techniques significantly improve the perfor- mance of a competitive reinforcement learning based extractive system, with the auxiliary loss being more powerful than pretraining. 1 Introduction Extractive summarization remains a simple and fast approach to produce summaries which are grammatical and accurately represent the source text. In the news domain, these systems are able to use a dominant signal: the position of a sentence in the source document. Due to journalistic con- ventions which place important information early in the articles, the lead sentences often contain key information. In this paper, we explore how sys- tems can look beyond this simple trend. Naturally, automatic systems have all along ex- ploited position cues in news as key indicators of important content (Schiffman et al., 2002; Hong and Nenkova, 2014; Liu, 2019). The ‘lead’ base- ∗Equal contribution. Lead-3: Bangladesh beat fellow World Cup quarter- finalists Pakistan by 79 runs in the first one-day inter- national in Dhaka. Tamim Iqbal and Mushfiqur Rahim scored centuries as Bangladesh made 329 for six and Pak- istan could only muster 250 in reply. Pakistan will have the chance to level the three-match series on Sunday when the second odi takes place in Mirpur. Reference: Bangladesh beat fellow World Cup quarter- finalists Pakistan by 79 runs. Tamim Iqbal and Mush- fiqur Rahim scored centuries for Bangladesh. Bangladesh made 329 for six and Pakistan could only muster 250 in reply. Pakistan will have the chance to level the three- match series on Sunday. Lead-3: Standing up for what you believe. What does it cost you? What do you gain? Reference: Indiana town’s Memories Pizza is shut down after online threat. Its owners say they’d refuse to cater a same-sex couple’s …
Content truncated to 500 words
Labeled Text
AbstractSentence position is a strong feature for news summarization, since the lead often (but not always) summarizes the key points of the article. In this paper, we show that recent neural systems excessively exploit this trend, which although powerful for many inputs, is also detrimental when summarizing documents where important content should be extracted from later parts of the article. We propose two techniques to make systems sensitive to the importance of content in different parts of the article. The first technique employs ‘unbiased’ data; i.e., randomly shuffled sentences of the source document, to pretrain the model. The second technique uses an auxiliary ROUGE-based loss that encourages the model to distribute importance scores throughout a document by mimicking sentence-level ROUGE scores on the training data. We show that these techniques significantly improve the performance of a competitive reinforcement learning based extractive system, with the auxiliary loss being more powerful than pretraining.
Publication Details
- Paper ID:
- 8898
- Year:
- 2019
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan