Using Knowledge to Facilitate Factoid Answer Pinpointing

Using Knowledge to Facilitate Factoid Answer Pinpointing Eduard Hovy, Ulf Hermjakob, Chin-Yew Lin, Deepak Ravichandran Information Sciences Institute University of Southern California 4676 Admiralty Way Marina del Rey, CA 90292-6695 USA {hovy,ulf,cyl,ravichan}@isi.edu Abstract In order to answer factoid questions, the Webclopedia QA system employs a range of knowledge resources. These include a QA Typology with answer patterns, WordNet, information about typical numerical answer ranges, and semantic relations identified by a robust parser, to filter out likely-looking but wrong candidate answers. This paper describes the knowledge resources and their impact on system performance. 1. Introduction The TREC evaluations of QA systems (Voorhees, 1999) require answers to be drawn from a given source corpus. Early QA systems used a simple filtering technique, question word density within a fixed n-word window, to pinpoint answers. Robust though this may be, the window method is not accurate enough. In response, factoid question answering systems • have evolved into two types: Use-Knowledge: extract query words from the input question, perform IR against the source corpus, possibly segment resulting documents, identify a set of segments containing likely answers, apply a set of heuristics that each consults a different source of knowledge to score each candidate, rank them, and select the best (Harabagiu et al., 2001; Hovy et al., 2001; Srihari and Li, 2000; Abney et al., 2000). • Use-the-Web: extract query words from the question, perform IR against the web, extract likely answer-bearing sentences, canonicalize the results, and select the most frequent answer(s). Then, for justification, locate examples of the answers in the source corpus (Brill et al., 2001; Buchholz, 2001). Of course, these techniques can be combined: the popularity ratings from Use-the-Web can also be applied as a filtering criterion (Clarke et al., 2001), or the knowledge resource heuristics can filter the web results. However, simply going to the web without using further knowledge (Brill et al., 2001) may return the web’s majority opinions on astrology, the killers of JFK, the cancerous effects of microwave ovens, etc.—fun but not altogether trustworthy. In this paper we describe the range of filtering techniques our system Webclopedia applies, from simplest to most sophisticated, and indicate their impact on the system. 2. Webclopedia Architecture As shown in Figure 1, Webclopedia adopts the Use-Knowledge architecture. Its modules are described in more detail in (Hovy et al., 2001; • Hovy et al., 1999): Question parsing: Using BBN’s IdentiFinder (Bikel et al., 1999), the CONTEX parser (Hermjakob, 1997) produces a syntactic-semantic analysis of the question and determines the QA type. • Query formation: Single- and multi-word units (content words) are extracted from the analysis, and WordNet synsets (Fellbaum, 1998) are used for query expansion. A series of Boolean queries of decreasing specificity is formed. • IR: The publicly available IR engine MG (Witten et al., 1994) returns the top-ranked N documents. • Selecting and ranking sentences: For each document, the most promising K sentences are located and scored using a formula that rewards word and phrase overlap with the question and its expanded …

Content truncated to 500 words