A Generative Blog Post Retrieval Model that Uses Query Expansion based on External Collections

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 1057–1065, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP A Generative Blog Post Retrieval Model that Uses Query Expansion based on External Collections Wouter Weerkamp w.weerkamp@uva.nl Krisztian Balog k.balog@uva.nl ISLA, University of Amsterdam Maarten de Rijke mdr@science.uva.nl Abstract User generated content is characterized by short, noisy documents, with many spelling errors and unexpected language usage. To bridge the vocabulary gap be- tween the user’s information need and documents in a specific user generated content environment, the blogosphere, we apply a form of query expansion, i.e., adding and reweighing query terms. Since the blogosphere is noisy, query expansion on the collection itself is rarely effective but external, edited collections are more suitable. We propose a generative model for expanding queries using external col- lections in which dependencies between queries, documents, and expansion doc- uments are explicitly modeled. Differ- ent instantiations of our model are dis- cussed and make different (in)dependence assumptions. Results using two exter- nal collections (news and Wikipedia) show that external expansion for retrieval of user generated content is effective; besides, conditioning the external collection on the query is very beneficial, and making can- didate expansion terms dependent on just the document seems sufficient. 1 Introduction One of the grand challenges in information re- trieval is to bridge the vocabulary gap between a user and her information need on the one hand and the relevant documents on the other (Baeza-Yates and Ribeiro-Neto, 1999). In the setting of blogs or other types of user generated content, bridging this gap becomes even more challenging. This has several causes: (i) the spelling errors, unusual, cre- ative or unfocused language usage resulting from the lack of top-down rules and editors in the con- tent creation process, and (ii) the (often) limited length of user generated documents. Query expansion, i.e., modifying the query by adding and reweighing terms, is an often used technique to bridge the vocabulary gap. In gen- eral, query expansion helps more queries than it hurts (Balog et al., 2008b; Manning et al., 2008). However, when working with user gener- ated content, expanding a query with terms taken from the very corpus in which one is searching tends to be less effective (Arguello et al., 2008a; Weerkamp and de Rijke, 2008b)—topic drift is a frequent phenomenon here. To be able to ar- rive at a richer representation of the user’s infor- mation need, while avoiding topic drift resulting from query expansion against user generated con- tent, various authors have proposed to expand the query against an external corpus, i.e., a corpus dif- ferent from the target (user generated) corpus from which documents need to be retrieved. Our aim in this paper is to define and evaluate generative models for expanding queries using ex- ternal collections. We propose a retrieval frame- work in which dependencies between queries, documents, and expansion documents are explic- itly modeled. We instantiate the framework in multiple ways by making …

Content truncated to 500 words