Online Sentence Novelty Scoring for Topical Document Streams

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 567–572, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Online Sentence Novelty Scoring for Topical Document Streams Sungjin Lee Yahoo Labs 229 West 43rd Street, New York, NY 10036, USA junion@yahoo-inc.com Abstract The enormous amount of information on the Internet has raised the challenge of highlight- ing new information in the context of already viewed content. This type of intelligent inter- face can save users time and prevent frustra- tion. Our goal is to scale up novelty detec- tion to large web properties like Google News and Yahoo News. We present a set of light- weight features for online novelty scoring and fast nonlinear feature transformation methods. Our experimental results on the TREC 2004 shared task datasets show that the proposed method is not only efficient but also very pow- erful, significantly surpassing the best system at TREC 2004. 1 Introduction The Internet supplies a wealth of news content with a corresponding problem: finding the right content for different users. Search engines are helpful if a user is looking for something specific that can be cast as a keyword query. If a user does not know what to look for, recommendation engines can make personal- ized suggestions for stories that may interest the user. But both types of systems frequently represent content that the user has already consumed, leading to delay and frustration. Consequently, identifying novel infor- mation has been an essential aspect of studies on news information retrieval. Newsjunkie (Gabrilovich et al., 2004), for instance, describes a system that personal- izes a newsfeed based on a measure of information nov- elty: the user can be presented custom tailored news feeds that are novel in the context of documents that have already been reviewed. This will spare the user from hunting through duplicate and redundant content for new nuggets of information. Identifying genuinely novel information is also an essential aspect of update summarization (Nenkova and McKeown, 2012; Gao et al., 2013; Guo et al., 2013; Wang and Li, 2010; Ben- tivogli et al., 2011). But the temporal dynamics of a document stream are not generally the focus. Novelty detection has also been studied in Topic Detection and Tracking field for the First Story Detection task (Allan, 2002; Karkali1 et al., 2013; Karkali et al., 2014; Tsai and Zhang, 2011) where the aim is to detect novel doc- uments given previously seen documents. In this paper, we examine a slightly different problem; we perform novelty detection at the sentence level to highlight sen- tences that contain novel information. The novelty track in TREC was designed to serve as a shared task for exactly this type of research: find- ing novel, on-topic sentences from a news stream (Har- man, 2002). There were four tasks in the novelty track but we only focus on task 2 in this paper: “given rele- vant sentences in all documents, identify all novel sen- tences.” The track changed slightly from year to …

Content truncated to 500 words