Improving Web Search Relevance with Semantic Features

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 648–657, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Improving Web Search Relevance with Semantic Features Yumao Lu Fuchun Peng Gilad Mishne Xing Wei Benoit Dumoulin Yahoo! Inc. 701 First Avenue Sunnyvale, CA, 94089 yumaol,fuchun,gilad,xwei,benoitd@yahoo-inc.com Abstract Most existing information retrieval (IR) systems do not take much advantage of natural language processing (NLP) tech- niques due to the complexity and limited observed effectiveness of applying NLP to IR. In this paper, we demonstrate that substantial gains can be obtained over a strong baseline using NLP techniques, if properly handled. We propose a frame- work for deriving semantic text matching features from named entities identified in Web queries; we then utilize these features in a supervised machine-learned ranking approach, applying a set of emerging ma- chine learning techniques. Our approach is especially useful for queries that contain multiple types of concepts. Comparing to a major commercial Web search engine, we observe a substantial 4% DCG5 gain over the affected queries. 1 Introduction Most existing IR models score documents pri- marily based on various term statistics. In tra- ditional models—from classic probabilistic mod- els (Croft and Harper, 1979; Fuhr, 1992), through vector space models (Salton et al., 1975; Narita and Ogawa, 2000), to well studied statistical lan- guage models (Ponte and Croft, 2000; Lafferty and Zhai, 2001)—these term statistics have been captured directly in the ranking formula. More re- cently, learning to rank approaches to IR (Fried- man, 2002) have become prominent; in these frameworks, that aim at learning a ranking func- tion from data, term statistics are often modeled as term matching features in a machine learning process. Traditional text matching features are mainly based on frequencies of n-grams of the user’s query in a variety of document sections, such as the document title, body text, anchor text, and so on. Global information such as frequency of term or term group in the corpus may also be used, as well as its combination with local statistics – pro- ducing relative scores such as tf · idf or BM25 scores (Robertson et al., 1995). Matching may be restricted to certain window sizes to enforce proximity, or may be more lenient, allowing un- ordered sequences and nonconsecutive sequences for a higher recall. Even before machine learning was applied to IR, NLP techniques such as Named Entity Recog- nition (NER), Part-of-Speech (POS) tagging, and parsing have been applied to both query model- ing and document indexing (Smeaton and van Ri- jsbergen, 1988; Narita and Ogawa, 2000; Sparck- Jones, 1999). For example, statistical concept language models generalize classic n-gram mod- els to concept n-gram model by enforcing query term proximity within each concept (Srikanth and Srihari, 2003). However, researchers have of- ten reported limited gains or even decreased per- formance when applying NLP to IR (Voorhees, 1999). Typically, concepts detected through NLP tech- niques either in the query or in documents are used as proximity constraints for text match- ing (Sparck-Jones, 1999), ignoring …

Content truncated to 500 words