Answer Extraction

Answer Extraction Steven Abney Michael Collins Amit Singhal AT&T Shannon Laboratory 180 Park Ave. Florharn Park, NJ 07932 {abney, mcollins,singhal}@research.att.corn Abstract Information retrieval systems have typically concen- trated on retrieving a set of documents which are rel- evant to a user's query. This paper describes a sys- tem that attempts to retrieve a much smaller section of text, namely, a direct answer to a user's question. The SMART IR system is used to extract a ranked set of passages that are relevant to the query. En- tities are extracted from these passages as potential answers to the question, and ranked for plausibility according to how well their type matches the query, and according to their frequency and position in the passages. The system was evaluated at the TREC-8 question answering track: we give results and error analysis on these queries. 1 Introduction In this paper, we describe and evaluate a question- answering system based on passage retrieval and entity-extraction technology. There has long been a concensus in the Informa- tion Retrieval (IR) community that natural language processing has little to offer for retrieval systems. Plausibly, this is creditable to the preeminence of ad hoc document retrieval as the task of interest in IR. However, there is a growing recognition of the lim- itations of ad hoc retrieval, both in the sense that current systems have reached the limit of achievable performance, and in the sense that users' informa- tion needs are often not well characterized by docu- ment retrieval. In many cases, a user has a question with a spe- cific answer, such as What city is it where the Euro- pean Parliament meets? or Who discovered Pluto? In such cases, ranked answers with links to support- ing documentation are much more useful than the ranked list of documents that standard retrieval en- gines produce. The ability to answer specific questions also pro- vides a foundation for addressing quantitative in- quiries such as How many times has the Fed raised interest rates this year? which can be interpreted as the cardinality of the set of answers to a specific question that happens to have multiple correct an- swers, like On what date did the Fed raise interest rates this year? We describe a system that extracts specific an- swers from a document collection. The system's per- formance was evaluated in the question-answering track that has been introduced this year at the TREC information-retrieval conference. The major points of interest are the following. • Comparison of the system's performance to a system that uses the same passage retrieval component, but no natural language process- ing, shows that NLP provides significant perfor- mance improvements on the question-answering task. • The system is designed to build on the strengths of both IR and NLP technologies. This makes for much more robustness than a pure NLP sys- tem would have, while affording much greater precision than a pure IR system would have. • The task is broken into subtasks that admit of independent development …

Content truncated to 500 words