Optimizing Story Link Detection is not Equivalent to Optimizing New Event Detection

Optimizing Story Link Detection is not Equivalent to Optimizing New Event Detection Ayman Farahat PARC 3333 Coyote Hill Rd Palo Alto, CA 94304 farahat@parc.com Francine Chen PARC 3333 Coyote Hill Rd Palo Alto, CA 94304 fchen@parc.com Thorsten Brants PARC 3333 Coyote Hill Rd Palo Alto, CA 94304 thorsten@brants.net Abstract Link detection has been regarded as a core technology for the Topic Detection and Tracking tasks of new event detection. In this paper we formulate story link detec- tion and new event detection as informa- tion retrieval task and hypothesize on the impact of precision and recall on both sys- tems. Motivated by these arguments, we introduce a number of new performance enhancing techniques including part of speech tagging, new similarity measures and expanded stop lists. Experimental re- sults validate our hypothesis. 1 Introduction Topic Detection and Tracking (TDT) research is sponsored by the DARPA Translingual Information Detection, Extraction, and Summarization (TIDES) program. The research has five tasks related to organizing streams of data such as newswire and broadcast news (Wayne, 2000): story segmentation, topic tracking, topic detection, new event detection (NED), and link detection (LNK). A link detection system detects whether two stories are “linked”, or discuss the same event. A story about a plane crash and another story about the funeral of the crash vic- tims are considered to be linked. In contrast, a story about hurricane Andrew and a story about hurricane Agnes are not linked because they are two different events. A new event detection system detects when a story discusses a previously unseen or “not linked” event. Link detection is considered to be a core tech- nology for new event detection and the other tasks. Several groups are performing research in the TDT tasks of link detection and new event detection. Based on their findings, we incorporated a number of their ideas into our baseline system. CMU (Yang et al., 1998) and UMass (Allan et al., 2000a) found that for new event detection it was better to com- pare a new story against all previously seen stories than to cluster previously seen stories and compare a new story against the clusters. CMU (Carbonell et al., 2001) found that NED results could be im- proved by developing separate models for different news sources to that could capture idiosyncrasies of different sources, which we also extended to link de- tection. UMass reported on adapting a tracking sys- tem for NED detection (Allan et al., 2000b). Allan et. al , (Allan et al., 2000b) developed a NED system based upon a tracking technology and showed that to achieve high-quality first story detection, tracking effectiveness must improve to a degree that experi- ence suggests is unlikely. In this paper, while we reach a similar conclusion as (Allan et al., 2000b) for LNK and NED systems , we give specific directions for improving each system separately. We compare the link detection and new event detection tasks and discuss ways in which we have observed that tech- niques developed for one task …

Content truncated to 500 words