Neural Embedding Language Models in Semantic Clustering of Web Search Results

Neural Embedding Language Models in Semantic Clustering of Web Search Results Andrey Kutuzov, Elizaveta Kuzmenko University of Oslo, National Research University Higher School of Economics andreku@ifi.uio.no, eakuzmenko 2@edu.hse.ru Abstract In this paper, a new approach towards semantic clustering of the results of ambiguous search queries is presented. We propose using distributed vector representations of words trained with the help of prediction-based neural embedding models to detect senses of search queries and to cluster search engine results page according to these senses. The words from titles and snippets together with semantic relationships between them form a graph, which is further partitioned into components related to different query senses. This approach to search engine results clustering is evaluated against a new manually annotated evaluation data set of Russian search queries. We show that in the task of semantically clustering search results, prediction-based models slightly but stably outperform traditional count-based ones, with the same training corpora. Keywords: distributional semantics, neural embeddings, search results clustering 1. Introduction Many user queries in web search are semantically ambigu- ous. For instance, the query B-52 can relate to a bomber, a hairstyle, a cocktail or a rock band. Thus, search engines strive to diversify their output and to present results that are related to as many query senses as possible. These results are as a rule not sorted by their relation to different ambiguous query senses and returned in the order of their relevance ranking, which for many of them seems to be almost equal. However, for the users’ convenience in the case of an ambiguous query it is often plausible to consolidate results related to one meaning. It is especially important in vertical or personalized search or when mixing results from different search engines into one stream. The present paper deals with the problem of semantic cluster- ing of a search engine results page (SERP), on the Russian language data. Research related to search results clustering (SRC) has a long history: arguably, main milestones in the last decade are (Bernardini et al., 2009) and (Marco and Navigli, 2013). We follow the latter’s graph-based approach, but augment it with word embedding algorithms, which has recently gained considerable traction in NLP. The formal description of the problem is: given an ambiguous search query and n top search results, cluster these results according to the cor- responding query senses. Note that we deal only with the problem of clustering itself, leaving ranking results within cluster and labeling clusters with human-readable labels for further research. It is also important to emphasize that we do not use full document texts: our only input is their titles and snippets as presented on the search engine results page. Our main contributions are: 1. releasing a publicly available human-annotated dataset1 for evaluation of semantic clustering for Rus- sian search engine results, complementing existing data sets for English; 2. evaluating the performance of prediction-based dis- tributional semantic models versus traditional count- 1http://ling.go.mail.ru/ruambient based models for this task, using query graph ap- proach. The paper is …

Content truncated to 500 words