Model Adaptation via Model Interpolation and Boosting for Web Search Ranking

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 505–513, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Model Adaptation via Model Interpolation and Boosting for Web Search Ranking Jianfeng Gao*, Qiang Wu*, Chris Burges*, Krysta Svore*, Yi Su#, Nazan Khan$, Shalin Shah$, Hongyan Zhou$ *Microsoft Research, Redmond, USA {jfgao; qiangwu; cburges; ksvore}@microsoft.com #Johns Hopkins University, USA suy@jhu.edu $Microsoft Bing Search, Redmond, USA {nazanka; a-shas; honzhou}@microsoft.com Abstract This paper explores two classes of model adapta- tion methods for Web search ranking: Model In- terpolation and error-driven learning approaches based on a boosting algorithm. The results show that model interpolation, though simple, achieves the best results on all the open test sets where the test data is very different from the training data. The tree-based boosting algorithm achieves the best performance on most of the closed test sets where the test data and the training data are sim- ilar, but its performance drops significantly on the open test sets due to the instability of trees. Several methods are explored to improve the robustness of the algorithm, with limited success. 1 Introduction We consider the task of ranking Web search results, i.e., a set of retrieved Web documents (URLs) are ordered by relevance to a query is- sued by a user. In this paper we assume that the task is performed using a ranking model (also called ranker for short) that is learned on labeled training data (e.g., human-judged query-document pairs). The ranking model acts as a function that maps the feature vector of a query-document pair to a real-valued score of relevance. Recent research shows that such a learned ranker is superior to classical retrieval models in two aspects (Burges et al., 2005; 2006; Gao et al., 2005). First, the ranking model can use arbitrary features. Both traditional criteria such as TF-IDF and BM25, and non-traditional features such as hyperlinks can be incorporated as features in the ranker. Second, if large amounts of high-quality human-judged query-document pairs were available for model training, the ranker could achieve significantly better retrieval results than the traditional retrieval models that cannot ben- efit from training data effectively. However, such training data is not always available for many search domains, such as non-English search markets or person name search. One of the most widely used strategies to re- medy this problem is model adaptation, which attempts to adjust the parameters and/or struc- ture of a model trained on one domain (called the background domain), for which large amounts of training data are available, to a different domain (the adaptation domain), for which only small amounts of training data are available. In Web search applications, domains can be defined by query types (e.g., person name queries), or lan- guages, etc. In this paper we investigate two classes of model adaptation methods for Web search ranking: Model Interpolation approaches and error-driven learning approaches. In model interpolation approaches, the adaptation data is used to derive a domain-specific model (also called in-domain model), which is then …

Content truncated to 500 words