Entailment due to Syntactically Encoded Semantic Relationships

Entailment due to Syntactically Encoded Semantic Relationships Elena Akhmatova Centre for Language Technology Macquarie University Sydney, Australia elena@ics.mq.edu.au Mark Dras Centre for Language Technology Macquarie University Sydney, Australia madras@ics.mq.edu.au Abstract The majority of the state-of-the-art ap- proaches to recognizing textual entailment focus on defining a generic approach to RTE. A generic approach never works well for every single entailment pair: there are entailment pairs that are recognized poorly by all the generic systems. Automatic iden- tification of such entailment pairs and apply- ing to them an RTE algorithm that is spe- cific to them could thus increase an overall performance of an entailment engine (that in this case will combine a generic RTE algo- rithm with a number of RTE algorithms for the problematic entailment pairs). We iden- tify one subtype of entailment pairs and de- velop a two-part probabilistic model for their classification into true and false entailments and evaluate it relative both to a baseline and to the RTE systems. We show that the model performs better than the baseline and the average of the systems from the RTE2 on both the balanced and unbalanced datasets we have created for evaluation. 1 Introduction Recognizing Textual Entailment (RTE) is a task where, given two text snippets, the goal is to deter- mine whether the meaning of one text snippet can be inferred from the meaning of the other (Dagan et al., 2005). The first of the text snippets in such a pair is referred to as the text and the other one as the hypothesis. The pair of text and hypothesis is called a text-hypothesis pair or entailment pair, with the two names considered to be synonymous. The text is usually much longer than the hypothesis. It can be represented by one or more coherent sen- tences, while the hypothesis is usually one short sen- tence. It is the meaning of the hypothesis that might or might not be entailed from the text. Thus, given a text-hypothesis pair, we recognize the relation be- tween the meanings of the text and the hypothesis in the pair as a true entailment if the meaning of the hypothesis is entailed from the meaning of the text. Otherwise, we recognize the relation between the meanings of the texts as a false entailment. There are several datasets for RTE. They contain text-hypothesis pairs marked yes if there is a relation of true entailment in a pair and no otherwise. These datasets are manually created annually for the RTE Challenges1 and are freely available. Most state-of-the-art approaches to RTE seek a generic approach to the task and do not differenti- ate between text-hypothesis pairs. However, a pos- sible alternative is to consider subclasses of entail- ment pairs and build models to handle these special- ties. An instance of this idea is proposed in Van- derwende and Dolan (2005), where the complete set of entailment pairs is divided in two: those whose categorization could be accurately predicted based solely on syntactic cues and those where it …

Content truncated to 500 words