Language Model Information Retrieval with Document Expansion

Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL, pages 407–414, New York, June 2006. c⃝2006 Association for Computational Linguistics Language Model Information Retrieval with Document Expansion Tao Tao, Xuanhui Wang, Qiaozhu Mei, ChengXiang Zhai Department of Computer Science University of Illinois at Urbana Champaign Abstract Language model information retrieval de- pends on accurate estimation of document models. In this paper, we propose a docu- ment expansion technique to deal with the problem of insufficient sampling of docu- ments. We construct a probabilistic neigh- borhood for each document, and expand the document with its neighborhood infor- mation. The expanded document provides a more accurate estimation of the docu- ment model, thus improves retrieval ac- curacy. Moreover, since document expan- sion and pseudo feedback exploit different corpus structures, they can be combined to further improve performance. The experi- ment results on several different data sets demonstrate the effectiveness of the pro- posed document expansion method. 1 Introduction Information retrieval with statistical language mod- els (Lafferty and Zhai, 2003) has recently attracted much more attention because of its solid theoreti- cal background as well as its good empirical per- formance. In this approach, queries and documents are assumed to be sampled from hidden generative models, and the similarity between a document and a query is then calculated through the similarity be- tween their underlying models. Clearly, good retrieval performance relies on the accurate estimation of the query and document mod- els. Indeed, smoothing of document models has been proved to be very critical (Chen and Good- man, 1998; Kneser and Ney, 1995; Zhai and Laf- ferty, 2001b). The need for smoothing originated from the zero count problem: when a term does not occur in a document, the maximum likelihood esti- mator would give it a zero probability. This is un- reasonable because the zero count is often due to in- sufficient sampling, and a larger sample of the data would likely contain the term. Smoothing is pro- posed to address the problem. While most smoothing methods utilize the global collection information with a simple interpolation (Ponte and Croft, 1998; Miller et al., 1999; Hiemstra and Kraaij, 1998; Zhai and Lafferty, 2001b), sev- eral recent studies (Liu and Croft, 2004; Kurland and Lee, 2004) have shown that local corpus structures can be exploited to improve retrieval performance. In this paper, we further study the use of local cor- pus structures for document model estimation and propose to use document expansion to better exploit local corpus structures for estimating document lan- guage models. According to statistical principles, the accuracy of a statistical estimator is largely determined by the sampling size of the observed data; a small data set generally would result in large variances, thus can not be trusted completely. Unfortunately, in re- trieval, we often have to estimate a model based on a single document. Since a document is a small sam- ple, our estimate is unlikely to be very accurate. A natural improvement is to enlarge …

Content truncated to 500 words