Can Click Patterns across User’s Query Logs Predict Answers to Definition Questions?

Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 99–108, Avignon, France, April 23 - 27 2012. c⃝2012 Association for Computational Linguistics Can Click Patterns across User’s Query Logs Predict Answers to Definition Questions? Alejandro Figueroa Yahoo! Research Latin America Blanco Encalada 2120, Santiago, Chile afiguero@yahoo-inc.com Abstract In this paper, we examined click patterns produced by users of Yahoo! search engine when prompting definition questions. Reg- ularities across these click patterns are then utilized for constructing a large and hetero- geneous training corpus for answer rank- ing. In a nutshell, answers are extracted from clicked web-snippets originating from any class of web-site, including Knowledge Bases (KBs). On the other hand, non- answers are acquired from redundant pieces of text across web-snippets. The effectiveness of this corpus was as- sessed via training two state-of-the-art models, wherewith answers to unseen queries were distinguished. These test- ing queries were also submitted by search engine users, and their answer candidates were taken from their respective returned web-snippets. This corpus helped both techniques to finish with an accuracy higher than 70%, and to predict over 85% of the answers clicked by users. In particular, our results underline the importance of non-KB training data. 1 Introduction It is a well-known fact that definition queries are very popular across users of commercial search engines (Rose and Levinson, 2004). The essen- tial characteristic of definition questions is their aim for discovering as much as possible descrip- tive information about the concept being defined (a.k.a. definiendum, pl. definienda). Some exam- ples of this kind of query include “Who is Ben- jamin Millepied?” and “Tell me about Bank of America”. It is a standard practice of definition ques- tion answering (QA) systems to mine KBs (e.g., online encyclopedias and dictionaries) for reli- able descriptive information on the definiendum (Sacaleanu et al., 2008). Normally, these pieces of information (i.e., nuggets) explain different facets of the definiendum (e.g., “ballet choreographer” and “born in Bordeaux”), and the main idea con- sists in projecting the acquired nuggets into the set of answer candidates afterwards. However, the performance of this category of method falls into sharp decline whenever few or no coverage is found across KBs (Zhang et al., 2005; Han et al., 2006). Put differently, this technique usually succeeds in discovering the most relevant facts about the most promiment sense of the definien- dum. But it often misses many pertinent nuggets, especially those that can be paraphrased in several ways; and/or those regarding ancillary senses of the definiendum, which are hardly found in KBs. As a means of dealing with this, current strate- gies try to construct general definition models inferred from a collection of definitions com- ing from the Internet or KBs (Androutsopoulos and Galanis, 2005; Xu et al., 2005; Han et al., 2006). To a great extent, models exploiting non- KB sources demand considerable annotation ef- forts, or when the data is obtained automatically, they benefit from empirical thresholds that ensure a certain degree of …

Content truncated to 500 words