Recognising Textual Entailment with Logical Inference

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 628–635, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Recognising Textual Entailment with Logical Inference Johan Bos School of Informatics University of Edinburgh 2 Buccleuch Place Edinburgh, EH8 9LW jbos@inf.ed.ac.uk Katja Markert School of Computing University of Leeds Woodhouse Lane Leeds, LS2 9JT markert@comp.leeds.ac.uk Abstract We use logical inference techniques for recognising textual entailment. As the performance of theorem proving turns out to be highly dependent on not read- ily available background knowledge, we incorporate model building, a technique borrowed from automated reasoning, and show that it is a useful robust method to approximate entailment. Finally, we use machine learning to combine these deep semantic analysis techniques with simple shallow word overlap; the resulting hy- brid model achieves high accuracy on the RTE testset, given the state of the art. Our results also show that the different tech- niques that we employ perform very dif- ferently on some of the subsets of the RTE corpus and as a result, it is useful to use the nature of the dataset as a feature. 1 Introduction Recognising textual entailment (RTE) is the task to find out whether some text T entails a hypothesis H. This task has recently been the focus of a challenge organised by the PASCAL network in 2004/5.1 In Example 1550 H follows from T whereas this is not the case in Example 731. 1All examples are from the corpus released as part of the RTE challenge. It is downloadable from http://www. pascal-network.org/Challenges/RTE/. The exam- ple numbers have also been kept. Each example is marked for entailment as TRUE if H follows from T and FALSE otherwise. The dataset is described in Section 4.1. Example: 1550 (TRUE) T: In 1998, the General Assembly of the Nippon Sei Ko Kai (Anglican Church in Japan) voted to accept female priests. H: The Anglican church in Japan approved the ordination of women. Example: 731 (FALSE) T: The city Tenochtitlan grew rapidly and was the center of the Aztec’s great empire. H: Tenochtitlan quickly spread over the island, marshes, and swamps. The recognition of textual entailment is without doubt one of the ultimate challenges for any NLP system: if it is able to do so with reasonable accu- racy, it is clearly an indication that it has some thor- ough understanding of how language works. Indeed, recognising entailment bears similarities to Turing’s famous test to assess whether machines can think, as access to different sources of knowledge and the ability to draw inferences seem to be among the pri- mary ingredients for an intelligent system. More- over, many NLP tasks have strong links to entail- ment: in summarisation, a summary should be en- tailed by the text; paraphrases can be seen as mutual entailment between T and H; in IE, the extracted in- formation should also be entailed by the text. In this paper, we discuss two methods for recog- nising textual entailment: a shallow method …

Content truncated to 500 words