Joint Annotation of Search Queries

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 102–111, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Joint Annotation of Search Queries Michael Bendersky Dept. of Computer Science University of Massachusetts Amherst, MA bemike@cs.umass.edu W. Bruce Croft Dept. of Computer Science University of Massachusetts Amherst, MA croft@cs.umass.edu David A. Smith Dept. of Computer Science University of Massachusetts Amherst, MA dasmith@cs.umass.edu Abstract Marking up search queries with linguistic an- notations such as part-of-speech tags, cap- italization, and segmentation, is an impor- tant part of query processing and understand- ing in information retrieval systems. Due to their brevity and idiosyncratic structure, search queries pose a challenge to existing NLP tools. To address this challenge, we propose a probabilistic approach for perform- ing joint query annotation. First, we derive a robust set of unsupervised independent an- notations, using queries and pseudo-relevance feedback. Then, we stack additional classi- fiers on the independent annotations, and ex- ploit the dependencies between them to fur- ther improve the accuracy, even with a very limited amount of available training data. We evaluate our method using a range of queries extracted from a web search log. Experimen- tal results verify the effectiveness of our ap- proach for both short keyword queries, and verbose natural language queries. 1 Introduction Automatic mark-up of textual documents with lin- guistic annotations such as part-of-speech tags, sen- tence constituents, named entities, or semantic roles is a common practice in natural language process- ing (NLP). It is, however, much less common in in- formation retrieval (IR) applications. Accordingly, in this paper, we focus on annotating search queries submitted by the users to a search engine. There are several key differences between user queries and the documents used in NLP (e.g., news articles or web pages). As previous research shows, these differences severely limit the applicability of standard NLP techniques for annotating queries and require development of novel annotation approaches for query corpora (Bergsma and Wang, 2007; Barr et al., 2008; Lu et al., 2009; Bendersky et al., 2010; Li, 2010). The most salient difference between queries and documents is their length. Most search queries are very short, and even longer queries are usually shorter than the average written sentence. Due to their brevity, queries often cannot be divided into sub-parts, and do not provide enough context for accurate annotations to be made using the stan- dard NLP tools such as taggers, parsers or chun- kers, which are trained on more syntactically coher- ent textual units. A recent analysis of web query logs by Bendersky and Croft (2009) shows, however, that despite their brevity, queries are grammatically diverse. Some queries are keyword concatenations, some are semi- complete verbal phrases and some are wh-questions. It is essential for the search engine to correctly an- notate the query structure, and the quality of these query annotations has been shown to be a crucial first step towards the development of reliable and robust query processing, representation and under- standing algorithms (Barr et …

Content truncated to 500 words