Multiple Similarity Measures and Source-Pair Information in Story Link Detection

Multiple Similarity Measures and Source-Pair Information in Story Link Detection Francine Chen Ayman Farahat Palo Alto Research Center 3333 Coyote Hill Rd. Palo Alto, CA 94304 � fchen, farahat � @parc.com, thorsten@brants.net Thorsten Brants Abstract State-of-the-art story link detection systems, that is, systems that determine whether two sto- ries are about the same event or linked, are usu- ally based on the cosine-similarity measured between two stories. This paper presents a method for improving the performanceof a link detection system by using a variety of simi- larity measures and using source-pair specific statistical information. The utility of a num- ber of different similarity measures, including cosine, Hellinger, Tanimoto, and clarity, both alone and in combination, was investigated. We also compared several machine learning techniques for combining the different types of information. The techniques investigated were SVMs, voting, and decision trees, each of which makes use of similarity and statisti- cal information differently. Our experimental results indicate that the combination of similar- ity measures and source-pair specific statistical information using an SVM provides the largest improvement in estimating whether two stories are linked; the resulting system was the best- performing link detection system at TDT-2002. 1 Introduction Story link detection, as defined in the Topic Detection and Tracking (TDT) competition sponsored by the DARPA TIDES program, is the task of determining whether two stories, such as news articles and/or radio broadcasts, are about the same event, or linked. In TDT an event is de- fined as “something that happens at some specific time and place” (TDT, 2002). For example, a story about a tor- nado in Kansas in May and another story about a tornado in Nebraska in June should not be classified as linked be- cause they are about different events, although they both fall under the same general “topic” of natural disasters. But a story about damage due to a tornado in Kansas and a story about the clean-up and repairs due to the same tor- nado in Kansas are considered linked events. In the TDT link detection task, a link detection sys- tem is given a sequence of time-ordered sets of stories, where each set is from one news source. The system can “look ahead” N source files from the current source file being processed when deciding whether the current pair is linked. Because the TDT link detection task is focused on streams of news stories, one of the primary differences between link detection and the more traditional IR catego- rization task is that new events occur relatively frequently and comparisons of interest are focused on events that are not known in advance. One consequence of this is that the best-performingsystems usually adapt to new input. Link detection is thought of as the basis for other event-based topic analysis tasks, such as topic tracking, topic detec- tion, and first-story detection (TDT, 2002). 2 Background and Related Work The DARPA TDT story link detection task requires iden- tifying pairs of linked stories. The original language of the stories are …

Content truncated to 500 words