Text Mining Based Query Expansion for Chinese IR

Text Mining Based Query Expansion for Chinese IR Zhihan Li Yue Xu Shlomo Geva Faculty of Information Technology Queensland University of Technology Brisbane, QLD 4001, Australia Z7.li@student.qut.edu.au {yue.xu,s.geva}@qut.edu.au Abstract Query expansion has long been suggested as a technique for dealing with word mismatch problem in information retrieval. In this paper, we describe a novel query expansion method which incorporates text mining techniques into query expansion for improving Chinese infor- mation retrieval performance. Unlike most of the existing query expansion strategies which generally select indexing terms from the top N retrieved documents and use them to expand the query, in our proposed method, we apply text mining techniques to find patterns from the retrieved documents which contain relevant terms to the query terms, then use these rele- vant terms which can be indexing terms or in- dexing term patterns to expand the query. The experiment with NTCIR-5 collection shows apparent improvement in both precision and recall. 1 Introduction The amazing growing speed in the number of Chi- nese Internet users indicates that building Chinese information retrieval systems is in great demand. This paper presents a method aimed at improving the performance of Chinese document retrieval. Unlike English text in which sentences are se- quences of words delimited by white spaces, in Chinese text, sentences are represented as strings of Chinese characters without separating spaces be- tween words. For Chinese information retrieval, the query is usually a set of Chinese words rather than a sequence of Chinese characters. For character based Chinese information retrieval, since the texts are not segmented, the retrieved documents which con- tain the character sequence of the query may not be relevant to the query since they may not contain the words in the query. Therefore, the quality of cha- racter based Chinese information retrieval is not satisfactory. On the other hand, a study has shown that the relationship between segmentation and re- trieval is in fact non-monotonic, that is, high preci- sion of segmentation alone may not improve retrieval performance (Peng, Huang, Schuurmans and Cercone 2002). In this paper we propose a Chi- nese information retrieval model which combines character based retrieval and word based ranking to achieve better performance. Another critical problem is that the original query may not represent adequately what a user wants. By appending additional terms to the original query, Query Expansion attempts to construct a richer ex- pression to better represent the user’s information need. Pseudo relevance feedback is a popular tech- nique for query expansion. The basic idea is to au- tomatically extract additional relevant terms from the top ranked documents in the initial result list. These terms are added to the original query, and the extended query is executed with the expectation of improved performance. In this paper, we propose a new approach to improving the performance of Chinese document retrieval by expanding queries with highly correlated segmented words, generated by using text mining techniques. The remainder of this paper is structured as follow. Section 2 briefly reviews some related …

Content truncated to 500 words