Improving Query Spelling Correction Using Web Search Results

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 181–189, Prague, June 2007. c⃝2007 Association for Computational Linguistics Improving Query Spelling Correction Using Web Search Results Qing Chen Natural Language Processing Lab Northeastern University Shenyang, Liaoning, China, 110004 chenqing@ics.neu.edu.cn Mu Li Microsoft Research Asia 5F Sigma Center Zhichun Road, Haidian District Beijing, China, 100080 muli@microsoft.com Ming Zhou Microsoft Research Asia 5F Sigma Center Zhichun Road, Haidian District Beijing, China, 100080 mingzhou@microsoft.com Abstract Traditional research on spelling correction in natural language processing and infor- mation retrieval literature mostly relies on pre-defined lexicons to detect spelling er- rors. But this method does not work well for web query spelling correction, because there is no lexicon that can cover the vast amount of terms occurring across the web. Recent work showed that using search query logs helps to solve this problem to some extent. However, such approaches cannot deal with rarely-used query terms well due to the data sparseness problem. In this paper, a novel method is proposed for use of web search results to improve the existing query spelling correction models solely based on query logs by leveraging the rich information on the web related to the query and its top-ranked candidate. Ex- periments are performed based on real- world queries randomly sampled from search engine’s daily logs, and the results show that our new method can achieve 16.9% relative F-measure improvement and 35.4% overall error rate reduction in comparison with the baseline method. 1 Introduction Nowadays more and more people are using Inter- net search engine to locate information on the web. Search engines take text queries that users type as input, and present users with information of ranked web pages related to users’ queries. During this process, one of the important factors that lead to poor search results is misspelled query terms. Ac- tually misspelled queries are rather commonly ob- served in query logs, as shown in previous investi- gations into the search engine’s log data that around 10%~15% queries were misspelled (Cucer- zan and Brill, 2004). Sometimes misspellings are due to simple typo- graphic errors such as teh for the. In many cases the spelling errors are more complicated cognitive errors such as camoflauge for camouflage. As a matter of fact, correct spelling is not always an easy task – even many Americans cannot exactly spell out California governor’s last name: Schwar- zenegger. A spelling correction tool can help im- prove users’ efficiency in the first case, but it is more useful in the latter since the users cannot fig- ure out the correct spelling by themselves. There has been a long history of general-purpose spelling correction research in natural language processing and information retrieval literature (Kukich, 1992), but its application to web search 181 query is still a new challenge. Although there are some similarities in correction candidate genera- tion and selection, these two settings are quite dif- ferent in one fundamental problem: How to deter- mine the …

Content truncated to 500 words