Answering Definition Questions with Multiple Knowledge Sources

Answering Definition Questions Using Multiple Knowledge Sources Wesley Hildebrandt, Boris Katz, and Jimmy Lin MIT Computer Science and Artificial Intelligence Laboratory 32 Vassar Street, Cambridge, MA 02139 {wes,boris,jimmylin}@csail.mit.edu Abstract Definition questions represent a largely unex- plored area of question answering—they are different from factoid questions in that the goal is to return as many relevant “nuggets” of information about a concept as possible. We describe a multi-strategy approach to an- swering such questions using a database con- structed offline with surface patterns, a Web- based dictionary, and an off-the-shelf docu- ment retriever. Results are presented from component-level evaluation and from an end- to-end evaluation of our implemented system at the TREC 2003 Question Answering Track. 1 Introduction To date, research in question answering has concentrated on factoid questions such as “Who was Abraham Lincoln married to?” The standard strategy for answering these questions using a textual corpus involves a combination of information retrieval and named-entity extraction tech- nology; see (Voorhees, 2002) for an overview. Factoid questions, however, represent but one facet of question answering, whose broader goal is to provide humans with intuitive information access using natural language. In contrast to factoid questions, the objective for “defi- nition” questions is to produce as many useful “nuggets” of information as possible. For example, the answer to “Who is Aaron Copland?” might include the following: American composer wrote ballets and symphonies born in Brooklyn, New York, in 1900 son of a Jewish immigrant American communist civil rights advocate Until recently, definition questions remained a largely unexplored area of question answering. Standard factoid question answering technology, designed to extract sin- gle answers, cannot be directly applied to this task. The solution to this interesting research challenge will draw from related fields such as information extraction, multi- document summarization, and answer fusion. In this paper, we present an approach to answering definition questions that combines knowledge from three sources. We present results from our own component analysis and the TREC 2003 Question Answering Track. 2 Answering Definition Questions Our first step in answering a definition question is to ex- tract the concept for which information is being sought— called the target term, or simply, the target. Once the tar- get term has been found, three techniques are employed to retrieve relevant nuggets: lookup in a database created from the AQUAINT corpus1, lookup in a Web dictionary followed by answer projection, and lookup directly in the AQUAINT corpus with an IR engine. Answers from the three different sources are then merged to produce the fi- nal system output. The following subsections describe each of these modules in greater detail. 2.1 Target Extraction We have developed a simple pattern-based parser to ex- tract the target term using regular expressions. If the nat- ural language question does not fit any of our patterns, the parser heuristically extracts the last sequence of capi- talized words in the question as the target. Our simple target extractor was tested on all definition questions from the TREC-9 and TREC-10 QA Track …

Content truncated to 500 words