Web-based List Question Answering

Web-Based List Question Answering Hui Yang, Tat-Seng Chua School of Computing National University of Singapore 3 Science Drive 2, 117543, Singapore yangh@lycos.co.uk,chuats@comp.nus.edu.sg Abstract While research on question answering has be- come popular in recent years, the problem of ef- ficiently locating a complete set of distinct answers to list questions in huge corpora or the Web is still far from being solved. This paper ex- ploits the wealth of freely available text and link structures on the Web to seek complete answers to list questions. We introduce our system, FADA, which relies on question parsing, web page classification/clustering, and content extrac- tion to find reliable distinct answers with high re- call. 1 Introduction The Text REtrieval Conference Series (TREC) has greatly encouraged Question Answering (QA) re- search in the recent years. The QA main task in the recent TREC-12 involved retrieving short concise answers to factoid and list questions, and answer nuggets for definition questions (Voorhees, 2003). The list task in TREC-12 required systems to as- semble a set of distinct and complete exact answers as responses to questions like “What are the brand names of Belgian chocolates?”. Unlike the questions in previous TREC conferences, TREC-12 list ques- tions did not specify a target number of instances to return but expected all answers contained in the cor- pus. Current QA systems (Harabagiu et al., 2003; Katz et al., 2003) usually extract a ranked list of fac- toid answers from the top returned documents by retrieval engines. This is actually the traditional way to find factoid answers. The only difference between answering list questions and factoid questions here is that list QA systems allow for multiple answers, whose scores are above a cut-off threshold. An analysis of the results of TREC-12 list QA systems (Voorhees, 2003) reveals that many of them severely suffer from two general problems: low re- call and non-distinctive answers. The median aver- age F1 performance of list runs was only 21.3% while the best performer could only achieve 39.6% (Table 1). This unsatisfactory performance exposes the limitation of using only traditional Information Retrieval and Natural Language Processing tech- niques to find an exhaustive set of factoid answers as compared to only one. TREC-12 Run Tag Avg F1 LCCmainS03 0.396 nusmml03r2 0.319 MITCSAIL03c 0.134 isi03a 0.118 BBN2003B 0.097 Average 0.213 Table 1: TREC-12 Top 5 Performers (Voorhees, 2003) In contrast to the traditional techniques, the Web is used extensively in systems to rally round factoid questions. QA researchers have explored a variety of uses of the Web, ranging from surface pattern min- ing (Ravichandran et al., 2002), query formulation (Yang et al., 2003), answer validation (Magnini et al., 2002), to directly finding answers on the Web by data redundancy analysis (Brill et al., 2001). These systems demonstrated that with the help of the Web they could generally boost baseline performance by 25%-30% (Lin 2002). The well-known redundancy-based approach iden- tifies the factoid answer as an N-gram appearing most frequently on the Web (Brill et al. 2001). This idea …

Content truncated to 500 words