Improving English and Chinese Ad-Hoc Retrieval: TIPSTER Text Phase 3 Final Report

IMPROVING ENGLISH AND CHINESE AD-HOC RETRIEVAL: TIPSTER TEXT PHASE 3 FINAL REPORT Kui-Lam Kwok Computer Science Department, Queens College, CUNY 65-30 Kissena Boulevard, Flushing, NY 11367 • email: kwok@ir.cs.qc.edu phone: (718) 997 3482/3500 ABSTRACT We investigated both English and Chinese ad-hoc information retrieval (IR). Part of our objectives is to study the use of term, phrasal and topical concept level evidence, either individually or in combination, to improve retrieval accuracy. For short queries, we studied five term level techniques that together lead to improvements over standard ad-hoc 2-stage retrieval some 20% to 40% for TREC5 & 6 experiments. For long queries, we studied linguistic phrases as evidence to re-rank outputs of term level retrieval. It brings small improvements in both TREC5 & 6 experiments, but needs further confirmation. We also investigated clustering of output documents from term level retrieval. Our aim is to separate relevant and irrelevant documents into different clusters, and to re- rank the output list by groups based on query and cluster-profile matching. Investigation is still on-going. For Chinese IR, many results were confirmed or discovered. For example, accurate word segmentation is not as important as first thought, but short-word segmentation is preferable to long-word (phrase). Simple bigram representation can give very good retrieval. A stopword list is not necessary; and presence of non-content terms does not hurt evaluation results much. One only needs screening out statistical stopwords of high frequency. Character indexing by itself is not competitive, but is useful for augmenting short-words or bigrams. Best results were obtained by combining retrievals of bigram and short-word with character representation. Chinese IR retums better precision than English, and it is not clear if this is a language-related, or collection-related phenomenon. 1. INTRODUCTION As increasing amounts of computer-readable texts are becoming available on the web or on CDROMs, text searching and detection has become an indispensable tool for information users and analysts of all walks of life. Up till the late 1980's, research in text retrieval has been mainly with small collections of a few thousand items. Since 1990, with the foresight of the TIPSTER and TREC programs, substantial progress has been made to advance the state-of-the-art in text detection and ad-hoc information retrieval (IR) methodologies. Examples include: availability, experimentation and uniform evaluation of gigabyte- size collections, term weighting improvements, 2-stage 'pseudo-feedback' retrieval strategy, recognition of difficulties of short queries versus long, use of phrases, treatment of foreign languages for multilingual retrieval, among others. This investigation builds upon previous findings to bring further advances in this field using our PIRCS system. We have participated in all past TREC experiments with consistently superior results. Since 1996, we have also participated in the TIPSTER Text Phase 3 program. This report serves to summarize work that has been done, and some of the important findings for both English and Chinese IR. Section 2 and 3 gives an overview of our PIRCS system and the 2-stage retrieval strategy. Section 4 presents our work for English ad-hoc retrieval employing term, phrasal and topical …

Content truncated to 500 words