Is It the Right Answer? Exploiting Web Redundancy for Answer Validation

Is It the Right Answer? Exploiting Web Redundancy for Answer Validation Bernardo Magnini, Matteo Negri, Roberto Prevete and Hristo Tanev ITC-Irst, Centro per la Ricerca Scientifica e Tecnologica [magnini,negri,prevete,tanev]@itc.it Abstract Answer Validation is an emerging topic in Question Answering, where open do- main systems are often required to rank huge amounts of candidate answers. We present a novel approach to answer valida- tion based on the intuition that the amount of implicit knowledge which connects an answer to a question can be quantitatively estimated by exploiting the redundancy of Web information. Experiments carried out on the TREC-2001 judged-answer collec- tion show that the approach achieves a high level of performance (i.e. 81% suc- cess rate). The simplicity and the effi- ciency of this approach make it suitable to be used as a module in Question Answer- ing systems. 1 Introduction Open domain question-answering (QA) systems search for answers to a natural language question either on the Web or in a local document collec- tion. Different techniques, varying from surface pat- terns (Subbotin and Subbotin, 2001) to deep seman- tic analysis (Zajac, 2001), are used to extract the text fragments containing candidate answers. Several systems apply answer validation techniques with the goal of filtering out improper candidates by check- ing how adequate a candidate answer is with re- spect to a given question. These approaches rely on discovering semantic relations between the ques- tion and the answer. As an example, (Harabagiu and Maiorano, 1999) describes answer validation as an abductive inference process, where an answer is valid with respect to a question if an explanation for it, based on background knowledge, can be found. Although theoretically well motivated, the use of se- mantic techniques on open domain tasks is quite ex- pensive both in terms of the involved linguistic re- sources and in terms of computational complexity, thus motivating a research on alternative solutions to the problem. This paper presents a novel approach to answer validation based on the intuition that the amount of implicit knowledge which connects an answer to a question can be quantitatively estimated by exploit- ing the redundancy of Web information. The hy- pothesis is that the number of documents that can be retrieved from the Web in which the question and the answer co-occur can be considered a significant clue of the validity of the answer. Documents are searched in the Web by means of validation pat- terns, which are derived from a linguistic process- ing of the question and the answer. In order to test this idea a system for automatic answer validation has been implemented and a number of experiments have been carried out on questions and answers pro- vided by the TREC-2001 participants. The advan- tages of this approach are its simplicity on the one hand and its efficiency on the other. Automatic techniques for answer validation are of great interest for the development of open do- main QA systems. The availability of a completely automatic evaluation procedure makes it feasible QA systems …

Content truncated to 500 words