Question Answering Using Integrated Information Retrieval and Information Extraction

Proceedings of NAACL HLT 2007, pages 532–539, Rochester, NY, April 2007. c⃝2007 Association for Computational Linguistics Question Answering using Integrated Information Retrieval and Information Extraction Barry Schiffman and Kathleen R. McKeown Department of Computer Science Columbia University New York, NY 10027 bschiff,kathy@cs.columbia.edu Ralph Grishman Department of Computer Science New York University New York, NY 10003 grishman@cs.nyu.edu James Allan University of Massachusetts Department of Computer Science Amherst, MA 01003 allan@cs.umass.edu Abstract This paper addresses the task of provid- ing extended responses to questions re- garding specialized topics. This task is an amalgam of information retrieval, topical summarization, and Information Extrac- tion (IE). We present an approach which draws on methods from each of these ar- eas, and compare the effectiveness of this approach with a query-focused summa- rization approach. The two systems are evaluated in the context of the prosecution queries like those in the DARPA GALE distillation evaluation. 1 Introduction As question-answering systems advance from han- dling factoid questions to more complex requests, they must be able to determine how much informa- tion to include while making sure that the informa- tion selected is indeed relevant. Unlike factoid ques- tions, there is no clear criterion that defines the kind of phrase that answers the question; instead, there may be many phrases that could make up an answer and it is often unclear in advance, how many. As system developers, our goal is to yield high recall without sacrificing precision. In response to questions about particular events of interest that can be enumerated in advance, it is pos- sible to perform a deeper semantic analysis focusing on the entities, relations, and sub-events of interest. On the other hand, the deeper analysis may be error- ful and will also not always provide complete cov- erage of the information relevant to the query. The challenge, therefore, is to blend a shallower, robust approach with the deeper approach in an effective way. In this paper, we show how this can be achieved through a synergistic combination of information re- trieval and information extraction. We interleave in- formation retrieval (IR) and response generation, us- ing IR in high precision mode in the first stage to return a small number of documents that are highly likely to be relevant. Information extraction of enti- ties and events within these documents is then used to pinpoint highly relevant sentences and associated words are selected to revise the query for a sec- ond pass of retrieval, improving recall. As part of this process, we approximate the relevant context by measuring the proximity of the target name in the query and extracted events. Our approach has been evaluated in the frame- work of the DARPA GALE1 program. One of the GALE evaluations involves responding to questions based on a set of question templates, ranging from broad questions like “Provide information on X”, where X is an organization, to questions focused on particular classes of events. For the experiments pre- sented here, we used the GALE program’s prosecu- tion class of …

Content truncated to 500 words