A Practical Passage-based Approach for Chinese Document Retrieval

1 A Practical Passage-based Approach for Chinese Document Retrieval Szu-Yuan Chi1, Chung-Li Hsiao1, Lee-Feng Chien1,2 1. Department of Information Management, National Taiwan University 2. Institute of Information Science, Academia Sinica, Taipei, Taiwan {r93036, r93042}@im.ntu.edu.tw and lfchien@iis.sinica.edu.tw Abstract. TF*IDF-based methods, as they are easily to implement, are widely accepted in information retrieval industry. It is interesting to investigate a feasible and practical technique to improve the retrieval performance of these conventional IR methods. In this paper, we are going to introduce a good alternative approach that uses passage-based ranking as the second stage of the retrieval process in them. 1. Introduction Previous research has shown that passage-level evidences can bring additional benefits to document retrieval when documents are long or span different subject areas. Callan (1994) addressed that the types of passages explored by researchers can be grouped into three classes: discourse, semantic, and window. Discourse passages are based upon textual discourse units (e.g. sentences, paragraphs and sections). Semantic passages are based upon the subject or content of the text. Window passages are based upon a number of words. Kaszkiel & Zobel (1997, 2001) also investigated the effectiveness on using passages for information retrieval. Their research showed that passages can be used in different ways. One is to provide a good basis for a question-and-answer style of information retrieval. Another approach is to use passages as proxies for documents, and documents are ranked according to similarities computed for their passages. It also addressed that fixed-length arbitrary passages of 150 words or more and starting at small interval so that the passages heavily overlap can give substantial empowerments in effectiveness, particularly for collections of long documents. Recently Liu (2002) further demonstrated that passages can be used effectively in a language modeling framework. They found passage retrieval based on language models can provide more reliable performance than retrieval based on full documents. The previous works have proven that passage-based retrieval can get better performance than document-based ones in different applications. It’s unfortunately that conventional passage-based retrieval most calculates documents ranks with the aids of passage-level indexing and single-stage processing. Although this can reduce the computing cost of passage-based retrieval, it is not flexible to consider the contextual effects of matched query terms in a passage and determine an appropriate weighting scheme through the access of indices. In some cases, it needs to analyze the content of document, for example, to determine if the occurrences of the matched query terms are appearing in a critical passage. Our research on passage retrieval is just at the beginning. Our long term research goal is to adopt sophisticated text analysis in combination with index-based ranking schemes to reach a balance between retrieval speed and accuracy. The purpose of this paper aims at developing a practical approach to improving conventional TF*IDF IR methods, without the involvement of using some sophisticated techniques such as query expansion and ontology-based ranking. The goal is not trivial. As users’ queries are often short, only a few conventional IR systems provide query expansion. …

Content truncated to 500 words