Examining the State-of-the-Art in News Timeline Summarization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1322–1334 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 1322 Examining the State-of-the-Art in News Timeline Summarization Demian Gholipour Ghalandari1,2 and Georgiana Ifrim1 1Insight Centre for Data Analytics, University College Dublin, Ireland 2Aylien Ltd., Dublin, Ireland {demian.gholipour|georgiana.ifrim}@insight-centre.org Abstract Previous work on automatic news timeline summarization (TLS) leaves an unclear pic- ture about how this task can generally be ap- proached and how well it is currently solved. This is mostly due to the focus on individual subtasks, such as date selection and date sum- marization, and to the previous lack of appro- priate evaluation metrics for the full TLS task. In this paper, we compare different TLS strate- gies using appropriate evaluation frameworks, and propose a simple and effective combina- tion of methods that improves over the state- of-the-art on all tested benchmarks. For a more robust evaluation, we also present a new TLS dataset, which is larger and spans longer time periods than previous datasets. The dataset will be made available at https://github. com/complementizer/news-tls. 1 Introduction Timelines of news events can be useful to condense long-ranging news topics and can help us under- stand how current major events follow from prior events. Timeline summarization (TLS) aims to au- tomatically create such timelines, i.e., temporally ordered time-stamped textual summaries of events focused on a particular topic. While TLS has been studied before, most works treat it as a combination of two individual subtasks, 1) date selection and 2) date summarization, and only focus on one of these at a time (Tran et al., 2013a,b, 2015b). How- ever, these subtasks are almost never evaluated in combination, which leaves an unclear picture of how well TLS is being solved in general. Further- more, previously used evaluation metrics for the date selection and timeline summarization tasks are not appropriate since they do not consider the temporal alignment in the evaluation. Just until recently, there were no established experimental settings and appropriate metrics for the full TLS task (Martschat and Markert, 2017, 2018). Date Summary 2001-11-29 Enron could cost Dutch group $195m 2001-11-30 1,100 UK jobs go in Enron collapse 2001-12-02 Barclays: Enron bankruptcy will not af- fect business 2002-01-15 As Enron scandal spreads, US starts to question cash for influence culture [...] 2004-07-08 Jury indicts Lay for inflating Enron earn- ings 2006-05-25 Former Enron bosses found guilty 2006-07-05 Enron founder Lay dies 2008-02-22 US prison beckons British bankers who got cosy with Enron Table 1: Excerpt of an automatically constructed time- line about the company Enron, using article headlines as summaries. The shaded parts indicate that the date or summary matches entries in a corresponding human- written ground-truth timeline. In this paper, we examine existing strategies for the full TLS task and how well they actually work. We identify three high-level approaches: 1) Di- rect summarization treats TLS like text summariza- tion, e.g., by selecting a small subset of sentences from a massive collection of news articles; 2) …

Content truncated to 500 words