Recognizing Textual Relatedness with Predicate-Argument Structures

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 784–792, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Recognizing Textual Relatedness with Predicate-Argument Structures Rui Wang Dept of Computational Linguistics Saarland University 66123 Saarbr¨ucken, Germany rwang@coli.uni-sb.de Yi Zhang Dept of Computational Linguistics Saarland University LT-Lab, DFKI GmbH D-66123 Saarbr¨ucken, Germany yzhang@coli.uni-sb.de Abstract In this paper, we first compare several strategies to handle the newly proposed three-way Recognizing Textual Entailment (RTE) task. Then we define a new mea- surement for a pair of texts, called Textual Relatedness, which is a weaker concept than semantic similarity or paraphrase. We show that an alignment model based on the predicate-argument structures using this measurement can help an RTE system to recognize the Unknown cases at the first stage, and contribute to the improvement of the overall performance in the RTE task. In addition, several heterogeneous lexical resources are tested, and different contri- butions from them are observed. 1 Introduction Recognizing Textual Entailment (RTE) (Dagan et al., 2006) is a task to detect whether one Hypoth- esis (H) can be inferred (or entailed) by a Text (T). Being a challenging task, it has been shown that it is helpful to applications like question an- swering (Harabagiu and Hickl, 2006). The recent research on RTE extends the two-way annotation into three-way1 2, making it even more difficult, but more linguistic-motivated. The straightforward strategy is to treat it as a three-way classification task, but the performance suffers a significant drop even when using the same classifier and the same feature model. In fact, it can also be dealt with as an extension to the traditional two-way classification, e.g., by identi- 1http://nlp.stanford.edu/RTE3-pilot/ 2http://www.nist.gov/tac/tracks/2008/ rte/rte.08.guidelines.html fying the Entailment (E) cases first and then fur- ther label the Contradiction (C) and Unknown (U) T-H pairs. Some other researchers also work on detecting negative cases, i.e. contradiction, in- stead of entailment (de Marneffe et al., 2008). However, according to our best knowledge, the detailed comparison between these strategies has not been fully explored, let alone the impact of the linguistic motivation behind the strategy selection. This paper will address this issue. Take the following example from the RTE-4 test set (Giampiccolo et al., 2009) as an example, T: At least five people have been killed in a head-on train collision in north-eastern France, while others are still trapped in the wreckage. All the victims are adults. H: A French train crash killed children. This is a pair of two contradicting texts, the mentioning of events (i.e. train crash) in both T and H are assumed to refer the same event3. In fact, the only contradicting part lies in the sec- ond sentence of T against H, that is, whether there are children among the victims. Therefore, this pair could also be classified as a Known (K) pair (=E∪C) against Unknown (U) pairs, instead of being classified as a Non-entailment (N) case (=C∪U) against E case in the traditional two-way annotation. Furthermore, many state-of-the-art RTE ap- proaches which …

Content truncated to 500 words