Answering What-Is Questions by Virtual Annotation

Answering What-Is Questions by Virtual Annotation John Prager IBM T.J. Watson Research Center Yorktown Heights, N.Y. 10598 (914) 784-6809 jprager@us.ibm.com Dragomir Radev University of Michigan Ann Arbor, MI 48109 (734) 615-5225 radev@umich.edu Krzysztof Czuba Carnegie-Mellon University Pittsburgh, PA 15213 (412) 268 6521 kczuba@cs.cmu.edu ABSTRACT We present the technique of Virtual Annotation as a specialization of Predictive Annotation for answering definitional What is questions. These questions generally have the property that the type of the answer is not given away by the question, which poses problems for a system which has to select answer strings from suggested passages. Virtual Annotation uses a combination of knowledge-based techniques using an ontology, and statistical techniques using a large corpus to achieve high precision. Keywords Question-Answering, Information Retrieval, Ontologies 1. INTRODUCTION Question Answering is gaining increased attention in both the commercial and academic arenas. While algorithms for general question answering have already been proposed, we find that such algorithms fail to capture certain subtleties of particular types of questions. We propose an approach in which different types of questions are processed using different algorithms. We introduce a technique named Virtual Annotation (VA) for answering one such type of question, namely the What is question. We have previously presented the technique of Predictive Annotation (PA) [Prager, 2000], which has proven to be an effective approach to the problem of Question Answering. The essence of PA is to index the semantic types of all entities in the corpus, identify the desired answer type from the question, search for passages that contain entities with the desired answer type as well as the other query terms, and to extract the answer term or phrase. One of the weaknesses of PA, though, has been in dealing with questions for which the system cannot determine the correct answer type required. We introduce here an extension to PA which we call Virtual Annotation and show it to be effective for those “What is/are (a/an) X” questions that are seeking hypernyms of X. These are a type of definition question, which other QA systems attempt to answer by searching in the document collection for textual clues similar to those proposed by [Hearst, 1998], that are characteristic of definitions. Such an approach does not use the strengths of PA and is not successful in the cases in which a deeper understanding of the text is needed in order to identify the defining term in question. We first give a brief description of PA. We look at a certain class of What is questions and describe our basic algorithm. Using this algorithm we develop the Virtual Annotation technique, and evaluate its performance with respect to both the standard TREC and our own benchmark. We demonstrate on two question sets that the precision improves from .15 and .33 to .78 and .83 with the addition of VA. 2. BACKGROUND For our purposes, a question-answering (QA) system is one which takes a well-formed user question and returns an appropriate answer phrase found in a body of text. …

Content truncated to 500 words