Improving the Effectiveness of Information Retrieval with Clustering and Fusion

109 Computational Linguistics and Chinese Language Processing Vol. 6, No. 1, February 2001, pp. 109-125 © Computational Linguistics Society of R.O.C. Improving the Effectiveness of Information Retrieval with Clustering and Fusion Jian Zhang*, Jianfeng Gao+, Ming Zhou**, Jiaxing Wang++ Abstract Fusion and clustering are two approaches to improving the effectiveness of information retrieval. In fusion, ranked lists are combined together by various means. The motivation is that different IR systems will complement each other, because they usually emphasize different query features when determining relevance and retrieve different sets of documents. In clustering, documents are clustered either before or after retrieval. The motivation is that similar documents tend to be relevant to the same query so that this approach is likely to retrieve more relevant documents by identifying clusters of similar documents. In this paper, we present a novel fusion technique that can be combined with clustering to achieve consistent improvements over conventional approaches. Our method involves three steps: (1) clustering similar documents, (2) re-ranking retrieval results, and (3) combining retrieval results. 1. Introduction In terms of the overall performance on a large query set, none of the typical IR systems outperform others substantially, while for each individual query, the performance that different systems achieve varies greatly [Voorhees 1997]. This observation leads to the idea of combining results obtained by different IR systems to improve overall performance. Fusion is a technique that combines retrieval results (or ranked lists) obtained by * This work was done while the author worked for Microsoft Research Asia as a visiting student. Department of Computer Science and Technology of Tsinghua University, China E- mail:ajian@s1000e.cs.tsinghua.edu.cn + Microsoft Research Asia E-mail:jfgao@microsoft.com ** Microsoft Research Asia E-mail:mingzhou@microsoft.com ++ Department of Computer Science and Technology of Tsinghua University, China E- mail:wjx@s1000e.cs.tsinghua.edu.cn 110 J. Zhang et al. different systems. However, conventional fusion techniques only consider retrieval results, while the information embedded in the document collection (e.g. the similarity between documents) is ignored. On the other hand, document clustering applies the structure of a document collection, but it usually considers each individual ranked list separately and is not able to take advantage of multiple ranked lists. In this paper, we present a novel fusion technique that can be combined with clustering. Given multiple retrieval results obtained by different IR systems, we first perform clustering on each ranked list and obtain a set of clusters. We then identify the clusters that contain the most relevant documents. Each of these clusters is evaluated based on a metric called reliability. Documents in reliable clusters are re-ranked. That is, we set higher scores for these documents. Finally, a conventional fusion method is applied to combine multiple retrieval results, which are re-ranked. Our experiments on the TREC-5 Chinese collection show that the above approach achieves consistent improvements over conventional approaches. The remainder of this paper is organized as follows. Section 2 gives a brief survey of related work. In Section 3, we describe our method in detail. In Section 4, a series of experiments are presented to …

Content truncated to 500 words