Automatic Answer Typing for How-Questions

Proceedings of NAACL HLT 2007, pages 516–523, Rochester, NY, April 2007. c⃝2007 Association for Computational Linguistics Automatic Answer Typing for How-Questions Christopher Pinchak and Shane Bergsma Department of Computing Science University of Alberta Edmonton, Alberta, T6G 2E8, Canada � pinchak,bergsma � @cs.ualberta.ca Abstract We introduce an answer typing strategy specific to quantifiable how questions. Us- ing the web as a data source, we auto- matically collect answer units appropri- ate to a given how-question type. Exper- imental results show answer typing with these units outperforms traditional fixed- category answer typing and other strate- gies based on the occurrences of numeri- cal entities in text. 1 Introduction Question answering (QA) systems are emerging as a viable means of obtaining specific information in the face of large availability. Answer typing is an important part of QA because it allows the system to greatly reduce the number of potential answers, using general knowledge of the answer form for a specific question. For example, for what, where, and who questions like “What is the capital of Canada?”, answer typing can filter the phrases which might be proposed as candidate answers, perhaps only identi- fying those textual entities known to be cities. We focus on answer typing for how-questions, a subset of questions which have received little spe- cific attention in the QA community. Rather than seeking an open-ended noun or verb phrase, how- questions often seek a numerical measurement ex- pressed in terms of a certain kind of unit, as in the following example: Example 1: “How heavy is a grizzly bear?” An answer typing system might expect answers to include units like kilograms, pounds, or tons. Enti- ties with inappropriate units, such as feet, meters, or honey pots, would be excluded as candidate answers. We specifically handle the subset of how- questions that we call how-adjective questions; that is, questions of the form “How adjective...?” such as Example 1. In particular, we do not address “how many” questions, which usually specify the units di- rectly following many, nor “how much” questions, which generally seek a monetary value. Hand-crafting a comprehensive list of units ap- propriate to many different adjectives is time- consuming and likely to miss important units. For example, an annotator might miss gigabytes for a measure of “how large.” Instead of compiling a list manually, we propose a means of automatically gen- erating lists of appropriate units for a number of real- world questions. How-adjective questions represent a significant portion of queries sent to search engines; of the 35 million queries in the AOL search query data set (Pass et al., 2006), over 11,000 are of the form “how adjective...” – close to one in every three thou- sand queries. Of those 11,000 queries, 152 different adjectives are used, ranging from the expected “how old” and “how far” to the obscure “how orwellian.” This high proportion of queries is especially strik- ing given that search engines provide little sup- port for answering how-adjective questions. Indeed, most IR systems work by keyword …

Content truncated to 500 words