COGEX: A Logic Prover for Question Answering

COGEX: A Logic Prover for Question Answering Dan Moldovan, Christine Clark, Sanda Harabagiu Language Computer Corporation Richardson, Texas 75080 moldovan@languagecomputer.com Steve Maiorano ATP Washington, DC 20505 Abstract Recent TREC results have demonstrated the need for deeper text understanding methods. This paper introduces the idea of automated reasoning applied to question answering and shows the feasibility of integrating a logic prover into a Question Answering system. The approach is to transform questions and answer passages into logic representations. World knowledge axioms as well as linguistic ax- ioms are supplied to the prover which renders a deep understanding of the relationship be- tween question text and answer text. Moreover, the trace of the proofs provide answer justifica- tions. The results show that the prover boosts the performance of the QA system on TREC questions by 30%. 1 Introduction Motivation In spite of significant advances made recently in the Question Answering technology, there still remain many problems to be solved. Some of these are: bridging the gap between question and answer words, pinpointing ex- act answers, taking into consideration syntactic and se- mantic roles of words, better answer ranking, answer jus- tification, and others. The recent TREC results (Voorhees 2002) have demonstrated that many performing systems reached a plateau; the systems ranked from 4th to 14th answered correctly between 38.4% to 24.8% of the total number of questions. It is clear that new ideas based on a deeper language understanding are necessary to push further the QA technology. In this paper we introduce one such novel idea, the use of automated reasoning in QA, and show that it is fea- sible, effective, and scalable. We have implemented a Logic Prover, called COGEX (from the permutation of the first two syllables of the verb excogitate) which uni- formly codifies the question and answer text, as well as world knowledge resources, in order to use its inference engine to verify and extract any lexical relationships be- tween the question and its candidate answers. Usefulness of a Logic Prover in QA COGEX captures the syntax-based relationships such as the syntactic objects, syntactic subjects, prepositional at- tachments, complex nominals, and adverbial/adjectival adjuncts provided by the logic representation of text. In addition to the logic representations of questions and can- didate answers, the QA Logic Prover needs world knowl- edge axioms to link question concepts to answer con- cepts. These axioms are provided by the WordNet glosses represented in logic forms. Additionally, the prover needs rewriting procedures for semantically equivalent lexical patterns. With this deep and intelligent representation, COGEX effectively and efficiently re-ranks candidate an- swers by their correctness, extracts the exact answer, and ultimately eliminates incorrect answers. In this way, the Logic Prover is a powerful tool in boosting the accuracy of the QA system. Moreover, the trace of a proof consti- tutes a justification for that answer. Technical challenges The challenges one faces when using automated reason- ing in the context of NLP include: logic representation of open text, need of world knowledge axioms, logic rep- …

Content truncated to 500 words