Context and Learning in Novelty Detection

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 716–723, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Context and Learning in Novelty Detection Barry Schiffman and Kathleen R. McKeown Department of Computer Science Columbia University New York, N.Y. {bschiff,kathy}@cs.columbia.edu Abstract We demonstrate the value of using con- text in a new-information detection sys- tem that achieved the highest precision scores at the Text Retrieval Conference’s Novelty Track in 2004. In order to de- termine whether information within a sen- tence has been seen in material read pre- viously, our system integrates information about the context of the sentence with novel words and named entities within the sentence, and uses a specialized learning algorithm to tune the system parameters. 1 Introduction New-information detection addresses two important problems in a society awash in more digital infor- mation than people can exploit. A novelty detection system could help people who are tracking an event in the news, where numerous sources present simi- lar material. It could also provide a way to organize summaries by focusing on the most recent informa- tion, much like an automated bulletin service. We envision that many types of users would find such a system valuable. Certainly analysts, busi- ness people, and anyone interested in current events, would benefit from being able to track news stories automatically, without repetition. Different news or- ganizations report on the same event, often working hard to make their reports look different from one another, whether or not they have new material to report. Our system would help readers to zero in on new information. In addition, a focus on new information provides a way of organizing a general summary. Our approach is unique in representing and main- taining the focus in discourse. The idea stems from the fact that novelty often comes in bursts, which is not surprising since the articles are composed of some number of smaller, coherent segments. Each segment is started by some kind of introductory pas- sage, and that is where we expect to find the novel words. Novel words are identified by comparing the current sentence’s words against a table of all words seen in the inputs to that point. They let us know whether the entire segment is likely to con- tain more novel material. Subsequent passages are likely to continue the novel discussion whether or not they contain novel words. They may contain pronomial references or other anaphoric references to the novel entity. Our long-term goal is to inte- grate the approach described in this paper into our larger new-information detector, a system that per- forms a more complicated syntactic analysis of the input texts and employs machine learning to classify passages as new or old. Meanwhile, we tested our focus-based approach at the Novelty Track at the Text Retrieval Confer- ence (TREC) in 2004. The Novelty Tracks in 2003 and in 2004 were divided into four tasks; Task 1 and Task 3 incorporate retrieval, requiring submis- …

Content truncated to 500 words