Methods for Using Textual Entailment in Open-Domain Question Answering

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 905–912, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Methods for Using Textual Entailment in Open-Domain Question Answering Sanda Harabagiu and Andrew Hickl Language Computer Corporation 1701 North Collins Boulevard Richardson, Texas 75080 USA sanda@languagecomputer.com Abstract Work on the semantics of questions has argued that the relation between a ques- tion and its answer(s) can be cast in terms of logical entailment. In this paper, we demonstrate how computational systems designed to recognize textual entailment can be used to enhance the accuracy of current open-domain automatic question answering (Q/A) systems. In our experi- ments, we show that when textual entail- ment information is used to either filter or rank answers returned by a Q/A system, accuracy can be increased by as much as 20% overall. 1 Introduction Open-Domain Question Answering (Q/A) sys- tems return a textual expression, identified from a vast document collection, as a response to a question asked in natural language. In the quest for producing accurate answers, the open-domain Q/A problem has been cast as: (1) a pipeline of linguistic processes pertaining to the processing of questions, relevant passages and candidate an- swers, interconnected by several types of lexico- semantic feedback (cf. (Harabagiu et al., 2001; Moldovan et al., 2002)); (2) a combination of language processes that transform questions and candidate answers in logic representations such that reasoning systems can select the correct an- swer based on their proofs (cf. (Moldovan et al., 2003)); (3) a noisy-channel model which selects the most likely answer to a question (cf. (Echi- habi and Marcu, 2003)); or (4) a constraint sat- isfaction problem, where sets of auxiliary ques- tions are used to provide more information and better constrain the answers to individual ques- tions (cf. (Prager et al., 2004)). While different in their approach, each of these frameworks seeks to approximate the forms of semantic inference that will allow them to identify valid textual answers to natural language questions. Recently, the task of automatically recog- nizing one form of semantic inference – tex- tual entailment – has received much attention from groups participating in the 2005 and 2006 PASCAL Recognizing Textual Entailment (RTE) Challenges (Dagan et al., 2005). 1 As currently defined, the RTE task requires sys- tems to determine whether, given two text frag- ments, the meaning of one text could be reason- ably inferred, or textually entailed, from the mean- ing of the other text. We believe that systems de- veloped specifically for this task can provide cur- rent question-answering systems with valuable se- mantic information that can be leveraged to iden- tify exact answers from ranked lists of candidate answers. By replacing the pairs of texts evaluated in the RTE Challenge with combinations of ques- tions and candidate answers, we expect that textual entailment could provide yet another mechanism for approximating the types of inference needed in order answer questions accurately. In this paper, we present three different methods for …

Content truncated to 500 words