Japanese Abbreviation Expansion with Query and Clickthrough Logs

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 410–419, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Japanese Abbreviation Expansion with Query and Clickthrough Logs Kei Uchiumi† Mamoru Komachi‡ Keigo Machinaga† Toshiyuki Maezawa† Toshinori Satou† Yoshinori Kobayashi† ∗ †Yahoo Japan Corporation Midtown Tower, 9-7-1 Akasaka, Minato-ku, Tokyo 107-6211, Japan {kuchiumi,kmachina,tmaezawa,toshsato}@yahoo-corp.jp ykobayas@google.com ‡Nara Institute of Science and Technology 8916-5 Takayama, Ikoma, Nara 630-0192, Japan komachi@is.naist.jp Abstract A novel reranking method has been de- veloped to refine web search queries. A label propagation algorithm was applied on a clickthrough graph, and the candi- dates were reranked using a query lan- guage model. Our method first enumer- ates query candidates with common land- ing pages with regard to the given query to create a clickthrough graph. Second, it calculates the likelihood of the candidates, using a language model generated from web search query logs. Finally, the can- didates are sorted by the score calculated from the likelihood and label propagation. As a result, high precision and coverage were achieved in the task of Japanese ab- breviation expansion, without using hand- crafted training data. 1 Introduction The query expansion technique has been widely used in recent web-search engines. Query expan- sion significantly improves recall in information retrieval operations. It uses a thesaurus or syn- onym dictionary to reformulate a query, or to cor- rect spelling errors in search queries. In the early days of the speller, the dictionary was manually compiled by lexicographers. How- ever, it is time consuming to construct a broad coverage dictionary, and domain knowledge is re- quired to achieve high quality. Moreover, the rapid growth of the web makes it even harder to main- tain an up-to-date dictionary for the web. ∗The work of Kobayashi were performed at Yahoo Japan. Current affiliation: Google Japan, Roppongi Hills Mori Tower, 6-10-1 Roppongi, Minato-ku, Tokyo 106-6126, Japan To alleviate this problem, web-search engines often exploit web search query logs to automati- cally generate a thesaurus. A web search query is a query that a web user types into a web search en- gine to find information. It is noisy and sometimes ambiguous to detect query intent, but it is a great way to create a fresh web dictionary at low cost. Hence, the web search queries are widely used in the NLP field. For instance, Hagiwara and Suzuki (2009) used them for a query alteration task, and Sekine and Suzuki (2007) leveraged them for ac- quiring semantic categories. More recently, web search clickthrough logs have been explored in the field of lexical acquisi- tion. A web clickthrough is the process of clicking a URL and going to the page it refers. This en- sures that the landing page is appropriate since the web user follows the hyperlink after checking the information displayed, such as ‘title’, ‘URL’, and ‘summary’ of their search. Two distinct queries landing on the same ‘URL’ are possibly input for the same purpose, meaning that they are likely to be related. In the NLP …

Content truncated to 500 words