Semantic v.s. Positions: Utilizing Balanced Proximity in Language Model Smoothing for Information Retrieval

International Joint Conference on Natural Language Processing, pages 507–515, Nagoya, Japan, 14-18 October 2013. Semantic v.s. Positions: Utilizing Balanced Proximity in Language Model Smoothing for Information Retrieval∗ Rui Yan†,♮, Han Jiang†,♮, Mirella Lapata‡, Shou-De Lin⋆, Xueqiang Lv⋄, and Xiaoming Li† ♮School of Electronics Engineering and Computer Science, Peking University ⋆Dept. of Computer Science and Information Engineering, National Taiwan University ‡Institute for Language, Cognition and Computation, University of Edinburgh ⋄Beijing Key Laboratory of Internet Culture and Digital Dissemination Research, BISTU {r.yan, billybob, lxq, lxm}@pku.edu.cn, mlap@inf.ed.ac.uk, sdlin@csie.ntu.edu.tw Abstract Work on information retrieval has shown that language model smoothing leads to more accurate estimation of document models and hence is crucial for achiev- ing good retrieval performance. Several smoothing methods have been proposed in the literature, using either semantic or po- sitional information. In this paper, we pro- pose a unified proximity-based framework to smooth language models, leveraging se- mantic and positional information simul- taneously in combination. The key idea is to project terms to positions where they originally do not exist (i.e., zero count), which is actually a word count propa- gation process. We achieve this projec- tion through two proximity-based density functions indicating semantic association and positional adjacency. We balance the effects of semantic and positional smooth- ing, and score a document based on the smoothed language model. Experiments on four standard TREC test collections show that our smoothing model is effec- tive for information retrieval and generally performs better than the state of the art. 1 Introduction Recently, statistical language models have at- tracted much attention in the information retrieval community due to their solid theoretical back- ground as well as their success in a variety of retrieval tasks (Ponte and Croft, 1998; Zhai and Lafferty, 2001b). Queries and documents are as- sumed to be sampled from hidden generative mod- els, and the similarity between a document and a query is then calculated through the similarity be- tween their underlying language models. Clearly, ∗This work was partially done when Rui Yan was an in- tern in Intel Center, National Taiwan University †Indicates equal contributions good retrieval performance relies on the accurate estimation of the query and document models. As queries are generally too short (Zhai and Lafferty, 2001a), the entire retrieval problem is essentially reduced to the problem of estimating a document language model (Lavrenko and Croft, 2001; Liu and Croft, 2004). Larger observed data generally allow people to establish a more accurate statistical model. Un- fortunately, in retrieval, we often have to esti- mate a model based on a small sample of data (e.g., a single document or only a few docu- ments). Therefore, given limited data sampling, a language model estimation sometimes encounters with the zero count problem: the maximum like- lihood estimator would give unseen terms a zero probability, which is not reliable because a larger sample of the data would likely contain the term. Language model smoothing is proposed to address the problem, and has been demonstrated to affect retrieval performance significantly (Zhai and Laf- ferty, …

Content truncated to 500 words