Selecting Query Term Alternations for Web Search by Exploiting Query Contexts

Proceedings of ACL-08: HLT, pages 148–155, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Selecting Query Term Alterations for Web Search by Exploiting Query Contexts Guihong Cao Stephen Robertson Jian-Yun Nie Dept. of Computer Science and Operations Research Microsoft Research at Cambridge Dept. of Computer Science and Operations Research University of Montreal, Canada Cambridge, UK University of Montreal, Canada caogui@iro.umontreal.ca ser@microsoft.com nie@iro.umontreal.ca Abstract Query expansion by word alterations (alterna- tive forms of a word) is often used in Web search to replace word stemming. This allows users to specify particular word forms in a query. However, if many alterations are added, query traffic will be greatly increased. In this paper, we propose methods to select only a few useful word alterations for query expansion. The selection is made according to the appropriateness of the alteration to the query context (using a bigram language model), or according to its expected impact on the retrieval effectiveness (using a regres- sion model). Our experiments on two TREC collections will show that both methods only select a few expansion terms, but the retrieval effectiveness can be improved significantly. 1 Introduction Word stemming is a basic NLP technique used in most of Information Retrieval (IR) systems. It transforms words into their root forms so as to in- crease the chance to match similar words/terms that are morphological variants. For example, with stemming, “controlling” can match “controlled” because both have the same root “control”. Most stemmers, such as the Porter stemmer (Porter, 1980) and Krovetz stemmer (Krovetz, 1993), deal with stemming by stripping word suffixes accord- ing to a set of morphological rules. Rule-based ap- proaches are intuitive and easy to implement. However, while in general, most words can be stemmed correctly; there is often erroneous stem- ming that unifies unrelated words. For instance, “jobs” is stemmed to “job” in both “find jobs in Apple” and “Steve Jobs at Apple”. This is particu- larly problematic in Web search, where users often use special or new words in their queries. A stan- dard stemmer such as Porter’s will wrongly stem them. To better determine stemming rules, Xu and Croft (1998) propose a selective stemming method based on corpus analysis. They refine the Porter stemmer by means of word clustering: words are first clustered according to their co-occurrences in the text collection. Only word variants belonging to the same cluster will be conflated. Despite this improvement, the basic idea of word stemming is to transform words in both doc- uments and queries to a standard form. Once this is done, there is no means for users to require a spe- cific word form in a query – the word form will be automatically transformed, otherwise, it will not match documents. This approach does not seem to be appropriate for Web search, where users often specify particular word forms in their queries. An example of this is a quoted query such as “Steve Jobs”, or “US Policy”. If documents are stemmed, many pages about job offerings or …

Content truncated to 500 words