A Confidence Model for Syntactically-Motivated Entailment Proofs

Proceedings of Recent Advances in Natural Language Processing, pages 455–462, Hissar, Bulgaria, 12-14 September 2011. A Confidence Model for Syntactically-Motivated Entailment Proofs Asher Stern Dept. of Computer Science Bar-Ilan University Ramat Gan, Israel astern7@gmail.com Ido Dagan Dept. of Computer Science Bar-Ilan University Ramat Gan, Israel dagan@cs.biu.ac.il Abstract This paper presents a novel method for recognizing textual entailment which de- rives the hypothesis from the text through a sequence of parse tree transforma- tions. Unlike related approaches based on tree-edit-distance, we employ trans- formations which better capture linguis- tic structures of entailment. This is achieved by (a) extending an earlier deter- ministic knowledge-based algorithm with syntactically-motivated on-the-fly trans- formations, and (b) by introducing an al- gorithm that uniformly learns costs for all types of transformations. Our evaluations and analysis support the validity of this ap- proach. 1 Introduction Recognizing Textual Entailment (RTE) is the task of determining whether a given textual statement (a hypothesis), H, can be inferred by a given text passage, T (Dagan et al., 2005). In recent years, the task has attracted considerable interest, with research evolving around the six RTE challenges, organized by PASCAL1 and later under the NIST Text Analysis Conference (TAC)2. While some of the proposed RTE systems employed quite shal- low and ad-hoc techniques, a few principled ap- proaches for modeling entailment inference began emerging as well. This paper focuses on an appealing approach attempted in several previous works, which, like most RTE systems, utilizes parse-based represen- tations of the text and hypothesis. Within this approach the parse-tree of H is explicitly gen- erated from that of T by applying a sequence of tree transformation operations. In analogy to 1http://pascallin.ecs.soton.ac.uk/Challenges/RTE/ 2http://www.nist.gov/tac/ logic, that sequence can be referred to as a proof, by which the target proposition, represented as a parse tree, is generated from the given text propo- sitions using appropriate proof steps. In one line of these works (Wang and Man- ning, 2010; Heilman and Smith, 2010; Mehdad and Magnini, 2009) the tree-transformation op- erations followed mostly traditional tree edit dis- tance operations, such as node insertion, deletion and substitution, and learned their costs according to the given RTE training data. As described in more detail in Section 2, these transformations do not necessarily capture the syntactic structure of entailment-preserving transformations. On the other hand, a rich inventory of knowledge-based operations was employed by Bar-Haim et al. (2007a). Their operations enable transforming complete sub-trees which do capture the syntactic structure of entailment inferences. Nevertheless, their work did not include a learn- ing component for estimating proof costs and their tree-transformations were based only on available knowledge resources, without providing on-the- fly operations that could compensate for some in- evitably missing knowledge. In this work we aim to combine the com- plementing advantages of the above mentioned works while filling in some missing gaps. We utilize knowledge-based sub-tree transformations, following (Bar-Haim et al., 2007a), but augment them with a set of on the fly transformations that correspond to syntactically-motivated entailment inferences, whose reliability can …

Content truncated to 500 words