Latent Document Re-Ranking

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1571–1580, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Latent Document Re-Ranking Dong Zhou1,2 Vincent Wade1 1. University of Dublin, Trinity College, Dublin 2, Ireland 2. School of Computer and Communication, Hunan University, Changsha, Hunan, China dongzhou1979@hotmail.com Vincent.Wade@cs.tcd.ie Abstract The problem of re-ranking initial retrieval re- sults exploring the intrinsic structure of docu- ments is widely researched in information re- trieval (IR) and has attracted a considerable amount of time and study. However, one of the drawbacks is that those algorithms treat queries and documents separately. Further- more, most of the approaches are predomi- nantly built upon graph-based methods, which may ignore some hidden information among the retrieval set. This paper proposes a novel document re- ranking method based on Latent Dirichlet Al- location (LDA) which exploits the implicit structure of the documents with respect to original queries. Rather than relying on graph- based techniques to identify the internal struc- ture, the approach tries to find the latent struc- ture of “topics” or “concepts” in the initial re- trieval set. Then we compute the distance be- tween queries and initial retrieval results based on latent semantic information deduced. Em- pirical results demonstrate that the method can comfortably achieve significant improvement over various baseline systems. 1 Introduction Consider a traditional IR problem, where there exists a set of documents 𝔻 in the collection. In response to an information need (as expressed in a query 𝑛���), the system determines a best fit be- tween the query and the documents and returns a list of retrieval results, sorted in a decreasing or- der of their relevancy. In practice, high precision at the top rankings of the returned results is of particular interest. Generally, there are two ways to automatically assist in achieving this ultimate goal after an initial retrieval process (Baeza- Yates and Ribeiro-Neto, 1999): document re- ranking and query expansion/re-weighting. Since the latter normally need a second round of re- trieval process, our method focuses on the docu- ment re-ranking approach. We will focus on ad- justing the ranking positions directly over initial retrieval results set 𝔻𝑖𝑛𝑖𝑡 . Recently, there is a trend of exploring the hid- den structure of documents to re-rank results. Some of the approaches represent the document entities as a connected graph 𝐺. It is usually con- structed by links inferred from the content in- formation as a nearest-neighbor graph. For ex- ample, Zhang et al. (2005) proposed an affinity ranking graph to re-rank search results by opti- mizing diversity and information richness. Kur- land and Lee (2005) introduced a structural re- ranking approach by exploiting asymmetric rela- tionships between documents induced by lan- guage models. Diaz (2005); Deng et al. (2009) use a family of semi-supervised machine learn- ing methods among documents graph con- structed by incorporating different evidences. However in this work we are more interested in adopting an automatic approach. There are two important factors that should be taken into account when …

Content truncated to 500 words