Improving Search Results Quality by Customizing Summary Lengths

Proceedings of ACL-08: HLT, pages 701–709, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Improving Search Results Quality by Customizing Summary Lengths Michael Kaisser University of Edinburgh 2 Buccleuch Place Edinburgh EH8 9LW m.kaisser@sms.ed.ac.uk Marti A. Hearst UC Berkeley 102 South Hall Berkeley, CA 94705 hearst@ischool.berkeley.edu John B. Lowe Powerset, Inc. 475 Brannan St. San Francisco, CA 94107 johnblowe@gmail.com Abstract Web search engines today typically show re- sults as a list of titles and short snippets that summarize how the retrieved documents are related to the query. However, recent research suggests that longer summaries can be prefer- able for certain types of queries. This pa- per presents empirical evidence that judges can predict appropriate search result summary lengths, and that perceptions of search result quality can be affected by varying these result lengths. These findings have important impli- cations for search results presentation, espe- cially for natural language queries. 1 Introduction Search results listings on the web have become stan- dardized as a list of information summarizing the retrieved documents. This summary information is often referred to as the document’s surrogate (Mar- chionini et al., 2008). In older search systems, such as those used in news and legal search, the document surrogate typ- ically consisted of the title and important metadata, such as date, author, source, and length of the article, as well as the document’s manually written abstract. In most cases, the full text content of the document was not available to the search engine and so no ex- tracts could be made. In web search, document surrogates typically show the web page’s title, a URL, and information extracted from the full text contents of the docu- ment. This latter part is referred to by several dif- ferent names, including summary, abstract, extract, and snippet. Today it is standard for web search en- gines to show these summaries as one or two lines of text, often with ellipses separating sentence frag- ments. However, there is evidence that the ideal re- sult length is often longer than the standard snippet length, and that furthermore, result length depends on the type of answer being sought. In this paper, we systematically examine the ques- tion of search result length preference, comparing different result lengths for different query types. We find evidence that desired answer length is sensitive to query type, and that for some queries longer an- swers are judged to be of higher quality. In the following sections we summarize the re- lated work on result length variation and on query topic classification. We then describe two studies. In the first, judges examined queries and made predic- tions about the expected answer types and the ideal answer lengths. In the second study, judges rated answers of different lengths for these queries. The studies find evidence supporting the idea that differ- ent query types are best answered with summaries of different lengths. 2 Related Work 2.1 Query-biased Summaries In the early days of the web, the result summary consisted of …

Content truncated to 500 words